{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configurations for Colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "IN_COLAB = 'google.colab' in sys.modules\n",
    "\n",
    "if IN_COLAB:\n",
    "    !apt-get install -y xvfb python-opengl > /dev/null 2>&1\n",
    "    !pip install gym pyvirtualdisplay > /dev/null 2>&1\n",
    "    !pip install JSAnimation==0.1\n",
    "    \n",
    "    from pyvirtualdisplay import Display\n",
    "    \n",
    "    # Start virtual display\n",
    "    dis = Display(visible=0, size=(400, 400))\n",
    "    dis.start()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 03. Prioritized Experience Replay (PER)\n",
    "\n",
    "[T. Schaul et al., \"Prioritized Experience Replay.\" arXiv preprint arXiv:1511.05952, 2015.](https://arxiv.org/pdf/1511.05952.pdf)\n",
    "\n",
    "Using a replay memory leads to design choices at two levels: which experiences to store, and which experiences to replay (and how to do so). This paper addresses only the latter: making the most effective use of the replay memory for learning, assuming that its contents are outside of our control.\n",
    "\n",
    "The central component of prioritized replay is the criterion by which the importance of each transition is measured. A reasonable approach is to use the magnitude of a transition’s TD error $\\delta$, which indicates how ‘surprising’\n",
    "or unexpected the transition is. This algorithm stores the last encountered TD error along with each transition in the replay memory. The transition with the largest absolute TD error is replayed from the memory. A Q-learning update\n",
    "is applied to this transition, which updates the weights in proportion to the TD error. One thing to note that new transitions arrive without a known TD-error, so it puts them at maximal priority in order to guarantee that all experience is seen at least once. (see *store* method)\n",
    "\n",
    "We might use 2 ideas to deal with TD-error: 1. greedy TD-error prioritization, 2. stochastic prioritization. However, greedy TD-error prioritization has a severe drawback. Greedy prioritization focuses on asmall subset of the experience: errors shrink slowly, especially when using function approximation, meaning that the initially high error transitions get replayed frequently. This lack of diversity that makes the system prone to over-fitting. To overcome this issue, we will use a stochastic sampling method that interpolates between pure greedy prioritization and uniform random sampling.\n",
    "\n",
    "$$\n",
    "P(i) = \\frac{p_i^{\\alpha}}{\\sum_k p_k^{\\alpha}}\n",
    "$$\n",
    "\n",
    "where $p_i > 0$ is the priority of transition $i$. The exponent $\\alpha$ determines how much prioritization is used, with $\\alpha = 0$ corresponding to the uniform case. In practice, we use additional term $\\epsilon$ in order to guarantee all transactions can be possibly sampled: $p_i = |\\delta_i| + \\epsilon$, where $\\epsilon$ is a small positive constant.\n",
    "\n",
    "One more. Let's recall one of the main ideas of DQN. To remove correlation of observations, it uses uniformly random sampling from the replay buffer. Prioritized replay introduces bias because it doesn't sample experiences uniformly at random due to the sampling proportion correspoding to TD-error. We can correct this bias by using importance-sampling (IS) weights\n",
    "\n",
    "$$\n",
    "w_i = \\big( \\frac{1}{N} \\cdot \\frac{1}{P(i)} \\big)^\\beta\n",
    "$$\n",
    "\n",
    "that fully compensates for the non-uniform probabilities $P(i)$ if $\\beta = 1$. These weights can be folded into the Q-learning update by using $w_i\\delta_i$ instead of $\\delta_i$. In typical reinforcement learning scenarios, the unbiased nature of the updates is most important near convergence at the end of training, We therefore exploit the flexibility of annealing the amount of importance-sampling correction over time, by defining a schedule on the exponent $\\beta$ that reaches 1 only at the end of learning. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import random\n",
    "from typing import Dict, List, Tuple\n",
    "\n",
    "import gym\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from IPython.display import clear_output\n",
    "from torch.nn.utils import clip_grad_norm_\n",
    "\n",
    "if IN_COLAB and not os.path.exists(\"segment_tree.py\"):\n",
    "    # download segment tree module\n",
    "    !wget https://raw.githubusercontent.com/curt-park/rainbow-is-all-you-need/master/segment_tree.py\n",
    "        \n",
    "from segment_tree import MinSegmentTree, SumSegmentTree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Replay buffer\n",
    "\n",
    "Please see *01.dqn.ipynb* for detailed description."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ReplayBuffer:\n",
    "    \"\"\"A simple numpy replay buffer.\"\"\"\n",
    "\n",
    "    def __init__(self, obs_dim: int, size: int, batch_size: int = 32):\n",
    "        self.obs_buf = np.zeros([size, obs_dim], dtype=np.float32)\n",
    "        self.next_obs_buf = np.zeros([size, obs_dim], dtype=np.float32)\n",
    "        self.acts_buf = np.zeros([size], dtype=np.float32)\n",
    "        self.rews_buf = np.zeros([size], dtype=np.float32)\n",
    "        self.done_buf = np.zeros(size, dtype=np.float32)\n",
    "        self.max_size, self.batch_size = size, batch_size\n",
    "        self.ptr, self.size, = 0, 0\n",
    "\n",
    "    def store(\n",
    "        self,\n",
    "        obs: np.ndarray,\n",
    "        act: np.ndarray, \n",
    "        rew: float, \n",
    "        next_obs: np.ndarray, \n",
    "        done: bool,\n",
    "    ):\n",
    "        self.obs_buf[self.ptr] = obs\n",
    "        self.next_obs_buf[self.ptr] = next_obs\n",
    "        self.acts_buf[self.ptr] = act\n",
    "        self.rews_buf[self.ptr] = rew\n",
    "        self.done_buf[self.ptr] = done\n",
    "        self.ptr = (self.ptr + 1) % self.max_size\n",
    "        self.size = min(self.size + 1, self.max_size)\n",
    "\n",
    "    def sample_batch(self) -> Dict[str, np.ndarray]:\n",
    "        idxs = np.random.choice(self.size, size=self.batch_size, replace=False)\n",
    "        return dict(obs=self.obs_buf[idxs],\n",
    "                    next_obs=self.next_obs_buf[idxs],\n",
    "                    acts=self.acts_buf[idxs],\n",
    "                    rews=self.rews_buf[idxs],\n",
    "                    done=self.done_buf[idxs])\n",
    "\n",
    "    def __len__(self) -> int:\n",
    "        return self.size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prioritized replay Buffer\n",
    "\n",
    "The key concept of PER's implementation is *Segment Tree*. It efficiently stores and samples transitions while managing the priorities of them. We recommend you understand how it works before you move on. Here are references for you:\n",
    "\n",
    "- In Korean: https://mrsyee.github.io/rl/2019/01/25/PER-sumtree/\n",
    "- In English: https://www.geeksforgeeks.org/segment-tree-set-1-sum-of-given-range/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PrioritizedReplayBuffer(ReplayBuffer):\n",
    "    \"\"\"Prioritized Replay buffer.\n",
    "    \n",
    "    Attributes:\n",
    "        max_priority (float): max priority\n",
    "        tree_ptr (int): next index of tree\n",
    "        alpha (float): alpha parameter for prioritized replay buffer\n",
    "        sum_tree (SumSegmentTree): sum tree for prior\n",
    "        min_tree (MinSegmentTree): min tree for min prior to get max weight\n",
    "        \n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(\n",
    "        self, \n",
    "        obs_dim: int,\n",
    "        size: int, \n",
    "        batch_size: int = 32, \n",
    "        alpha: float = 0.6\n",
    "    ):\n",
    "        \"\"\"Initialization.\"\"\"\n",
    "        assert alpha >= 0\n",
    "        \n",
    "        super(PrioritizedReplayBuffer, self).__init__(obs_dim, size, batch_size)\n",
    "        self.max_priority, self.tree_ptr = 1.0, 0\n",
    "        self.alpha = alpha\n",
    "        \n",
    "        # capacity must be positive and a power of 2.\n",
    "        tree_capacity = 1\n",
    "        while tree_capacity < self.max_size:\n",
    "            tree_capacity *= 2\n",
    "\n",
    "        self.sum_tree = SumSegmentTree(tree_capacity)\n",
    "        self.min_tree = MinSegmentTree(tree_capacity)\n",
    "        \n",
    "    def store(\n",
    "        self, \n",
    "        obs: np.ndarray, \n",
    "        act: int, \n",
    "        rew: float, \n",
    "        next_obs: np.ndarray, \n",
    "        done: bool\n",
    "    ):\n",
    "        \"\"\"Store experience and priority.\"\"\"\n",
    "        super().store(obs, act, rew, next_obs, done)\n",
    "        \n",
    "        self.sum_tree[self.tree_ptr] = self.max_priority ** self.alpha\n",
    "        self.min_tree[self.tree_ptr] = self.max_priority ** self.alpha\n",
    "        self.tree_ptr = (self.tree_ptr + 1) % self.max_size\n",
    "\n",
    "    def sample_batch(self, beta: float = 0.4) -> Dict[str, np.ndarray]:\n",
    "        \"\"\"Sample a batch of experiences.\"\"\"\n",
    "        assert len(self) >= self.batch_size\n",
    "        assert beta > 0\n",
    "        \n",
    "        indices = self._sample_proportional()\n",
    "        \n",
    "        obs = self.obs_buf[indices]\n",
    "        next_obs = self.next_obs_buf[indices]\n",
    "        acts = self.acts_buf[indices]\n",
    "        rews = self.rews_buf[indices]\n",
    "        done = self.done_buf[indices]\n",
    "        weights = np.array([self._calculate_weight(i, beta) for i in indices])\n",
    "        \n",
    "        return dict(\n",
    "            obs=obs,\n",
    "            next_obs=next_obs,\n",
    "            acts=acts,\n",
    "            rews=rews,\n",
    "            done=done,\n",
    "            weights=weights,\n",
    "            indices=indices,\n",
    "        )\n",
    "        \n",
    "    def update_priorities(self, indices: List[int], priorities: np.ndarray):\n",
    "        \"\"\"Update priorities of sampled transitions.\"\"\"\n",
    "        assert len(indices) == len(priorities)\n",
    "\n",
    "        for idx, priority in zip(indices, priorities):\n",
    "            assert priority > 0\n",
    "            assert 0 <= idx < len(self)\n",
    "\n",
    "            self.sum_tree[idx] = priority ** self.alpha\n",
    "            self.min_tree[idx] = priority ** self.alpha\n",
    "\n",
    "            self.max_priority = max(self.max_priority, priority)\n",
    "            \n",
    "    def _sample_proportional(self) -> List[int]:\n",
    "        \"\"\"Sample indices based on proportions.\"\"\"\n",
    "        indices = []\n",
    "        p_total = self.sum_tree.sum(0, len(self) - 1)\n",
    "        segment = p_total / self.batch_size\n",
    "        \n",
    "        for i in range(self.batch_size):\n",
    "            a = segment * i\n",
    "            b = segment * (i + 1)\n",
    "            upperbound = random.uniform(a, b)\n",
    "            idx = self.sum_tree.retrieve(upperbound)\n",
    "            indices.append(idx)\n",
    "            \n",
    "        return indices\n",
    "    \n",
    "    def _calculate_weight(self, idx: int, beta: float):\n",
    "        \"\"\"Calculate the weight of the experience at idx.\"\"\"\n",
    "        # get max weight\n",
    "        p_min = self.min_tree.min() / self.sum_tree.sum()\n",
    "        max_weight = (p_min * len(self)) ** (-beta)\n",
    "        \n",
    "        # calculate weights\n",
    "        p_sample = self.sum_tree[idx] / self.sum_tree.sum()\n",
    "        weight = (p_sample * len(self)) ** (-beta)\n",
    "        weight = weight / max_weight\n",
    "        \n",
    "        return weight"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Network\n",
    "\n",
    "We are going to use a simple network architecture with three fully connected layers and two non-linearity functions (ReLU)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Network(nn.Module):\n",
    "    def __init__(self, in_dim: int, out_dim: int):\n",
    "        \"\"\"Initialization.\"\"\"\n",
    "        super(Network, self).__init__()\n",
    "\n",
    "        self.layers = nn.Sequential(\n",
    "            nn.Linear(in_dim, 128), \n",
    "            nn.ReLU(),\n",
    "            nn.Linear(128, 128), \n",
    "            nn.ReLU(), \n",
    "            nn.Linear(128, out_dim)\n",
    "        )\n",
    "\n",
    "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n",
    "        \"\"\"Forward method implementation.\"\"\"\n",
    "        return self.layers(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DQN + PER Agent\n",
    "\n",
    "Here is a summary of DQNAgent class.\n",
    "\n",
    "| Method           | Note                                                 |\n",
    "| ---              | ---                                                  |\n",
    "|select_action     | select an action from the input state.               |\n",
    "|step              | take an action and return the response of the env.   |\n",
    "|compute_dqn_loss  | return dqn loss.                                     |\n",
    "|update_model      | update the model by gradient descent.                |\n",
    "|target_hard_update| hard update from the local model to the target model.|\n",
    "|train             | train the agent during num_frames.                   |\n",
    "|test              | test the agent (1 episode).                          |\n",
    "|plot              | plot the training progresses.                        |\n",
    "\n",
    "\n",
    "All differences from pure DQN are noted with comments - PER.\n",
    "\n",
    "#### __init__\n",
    "\n",
    "Here, we use PrioritizedReplayBuffer, instead of ReplayBuffer, and use hold 2 more parameters beta and priority epsilon which arre used to calculate weights and new priorities respectively.\n",
    "\n",
    "#### compute_dqn_loss & update_model\n",
    "\n",
    "It returns every loss per each sample for importance sampling before average. After updating the nework, it is necessary to update priorities of all sampled experiences.\n",
    "\n",
    "#### train\n",
    "\n",
    "beta linearly increases to 1 at every training step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DQNAgent:\n",
    "    \"\"\"DQN Agent interacting with environment.\n",
    "    \n",
    "    Attribute:\n",
    "        env (gym.Env): openAI Gym environment\n",
    "        memory (ReplayBuffer): replay memory to store transitions\n",
    "        batch_size (int): batch size for sampling\n",
    "        epsilon (float): parameter for epsilon greedy policy\n",
    "        epsilon_decay (float): step size to decrease epsilon\n",
    "        max_epsilon (float): max value of epsilon\n",
    "        min_epsilon (float): min value of epsilon\n",
    "        target_update (int): period for target model's hard update\n",
    "        gamma (float): discount factor\n",
    "        dqn (Network): model to train and select actions\n",
    "        dqn_target (Network): target model to update\n",
    "        optimizer (torch.optim): optimizer for training dqn\n",
    "        transition (list): transition information including \n",
    "                           state, action, reward, next_state, done\n",
    "        beta (float): determines how much importance sampling is used\n",
    "        prior_eps (float): guarantees every transition can be sampled\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(\n",
    "        self, \n",
    "        env: gym.Env,\n",
    "        memory_size: int,\n",
    "        batch_size: int,\n",
    "        target_update: int,\n",
    "        epsilon_decay: float,\n",
    "        max_epsilon: float = 1.0,\n",
    "        min_epsilon: float = 0.1,\n",
    "        gamma: float = 0.99,\n",
    "        # PER parameters\n",
    "        alpha: float = 0.6,\n",
    "        beta: float = 0.4,\n",
    "        prior_eps: float = 1e-6,\n",
    "    ):\n",
    "        \"\"\"Initialization.\n",
    "        \n",
    "        Args:\n",
    "            env (gym.Env): openAI Gym environment\n",
    "            memory_size (int): length of memory\n",
    "            batch_size (int): batch size for sampling\n",
    "            target_update (int): period for target model's hard update\n",
    "            epsilon_decay (float): step size to decrease epsilon\n",
    "            lr (float): learning rate\n",
    "            max_epsilon (float): max value of epsilon\n",
    "            min_epsilon (float): min value of epsilon\n",
    "            gamma (float): discount factor\n",
    "            alpha (float): determines how much prioritization is used\n",
    "            beta (float): determines how much importance sampling is used\n",
    "            prior_eps (float): guarantees every transition can be sampled\n",
    "        \"\"\"\n",
    "        obs_dim = env.observation_space.shape[0]\n",
    "        action_dim = env.action_space.n\n",
    "        \n",
    "        self.env = env\n",
    "        \n",
    "        self.batch_size = batch_size\n",
    "        self.epsilon = max_epsilon\n",
    "        self.epsilon_decay = epsilon_decay\n",
    "        self.max_epsilon = max_epsilon\n",
    "        self.min_epsilon = min_epsilon\n",
    "        self.target_update = target_update\n",
    "        self.gamma = gamma\n",
    "        \n",
    "        # device: cpu / gpu\n",
    "        self.device = torch.device(\n",
    "            \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "        )\n",
    "        print(self.device)\n",
    "        \n",
    "        # PER\n",
    "        # In DQN, We used \"ReplayBuffer(obs_dim, memory_size, batch_size)\"\n",
    "        self.beta = beta\n",
    "        self.prior_eps = prior_eps\n",
    "        self.memory = PrioritizedReplayBuffer(\n",
    "            obs_dim, memory_size, batch_size, alpha\n",
    "        )\n",
    "\n",
    "        # networks: dqn, dqn_target\n",
    "        self.dqn = Network(obs_dim, action_dim).to(self.device)\n",
    "        self.dqn_target = Network(obs_dim, action_dim).to(self.device)\n",
    "        self.dqn_target.load_state_dict(self.dqn.state_dict())\n",
    "        self.dqn_target.eval()\n",
    "        \n",
    "        # optimizer\n",
    "        self.optimizer = optim.Adam(self.dqn.parameters())\n",
    "\n",
    "        # transition to store in memory\n",
    "        self.transition = list()\n",
    "        \n",
    "        # mode: train / test\n",
    "        self.is_test = False\n",
    "\n",
    "    def select_action(self, state: np.ndarray) -> np.ndarray:\n",
    "        \"\"\"Select an action from the input state.\"\"\"\n",
    "        # epsilon greedy policy\n",
    "        if self.epsilon > np.random.random():\n",
    "            selected_action = self.env.action_space.sample()\n",
    "        else:\n",
    "            selected_action = self.dqn(\n",
    "                torch.FloatTensor(state).to(self.device)\n",
    "            ).argmax()\n",
    "            selected_action = selected_action.detach().cpu().numpy()\n",
    "        \n",
    "        if not self.is_test:\n",
    "            self.transition = [state, selected_action]\n",
    "        \n",
    "        return selected_action\n",
    "\n",
    "    def step(self, action: np.ndarray) -> Tuple[np.ndarray, np.float64, bool]:\n",
    "        \"\"\"Take an action and return the response of the env.\"\"\"\n",
    "        next_state, reward, done, _ = self.env.step(action)\n",
    "\n",
    "        if not self.is_test:\n",
    "            self.transition += [reward, next_state, done]\n",
    "            self.memory.store(*self.transition)\n",
    "    \n",
    "        return next_state, reward, done\n",
    "\n",
    "    def update_model(self) -> torch.Tensor:\n",
    "        \"\"\"Update the model by gradient descent.\"\"\"\n",
    "        # PER needs beta to calculate weights\n",
    "        samples = self.memory.sample_batch(self.beta)\n",
    "        weights = torch.FloatTensor(\n",
    "            samples[\"weights\"].reshape(-1, 1)\n",
    "        ).to(self.device)\n",
    "        indices = samples[\"indices\"]\n",
    "\n",
    "        # PER: importance sampling before average\n",
    "        elementwise_loss = self._compute_dqn_loss(samples)\n",
    "        loss = torch.mean(elementwise_loss * weights)\n",
    "\n",
    "        self.optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        # gradient clipping\n",
    "        # https://pytorch.org/docs/stable/nn.html#torch.nn.utils.clip_grad_norm_\n",
    "        clip_grad_norm_(self.dqn.parameters(), 1.0, norm_type=1)\n",
    "        self.optimizer.step()\n",
    "        \n",
    "        # PER: update priorities\n",
    "        loss_for_prior = elementwise_loss.detach().cpu().numpy()\n",
    "        new_priorities = loss_for_prior + self.prior_eps\n",
    "        self.memory.update_priorities(indices, new_priorities)\n",
    "\n",
    "        return loss.item()\n",
    "        \n",
    "    def train(self, num_frames: int, plotting_interval: int = 200):\n",
    "        \"\"\"Train the agent.\"\"\"\n",
    "        self.is_test = False\n",
    "        \n",
    "        state = self.env.reset()\n",
    "        update_cnt = 0\n",
    "        epsilons = []\n",
    "        losses = []\n",
    "        scores = []\n",
    "        score = 0\n",
    "\n",
    "        for frame_idx in range(1, num_frames + 1):\n",
    "            action = self.select_action(state)\n",
    "            next_state, reward, done = self.step(action)\n",
    "\n",
    "            state = next_state\n",
    "            score += reward\n",
    "            \n",
    "            # PER: increase beta\n",
    "            fraction = min(frame_idx / num_frames, 1.0)\n",
    "            self.beta = self.beta + fraction * (1.0 - self.beta)\n",
    "\n",
    "            # if episode ends\n",
    "            if done:\n",
    "                state = env.reset()\n",
    "                scores.append(score)\n",
    "                score = 0\n",
    "\n",
    "            # if training is ready\n",
    "            if len(self.memory) >= self.batch_size:\n",
    "                loss = self.update_model()\n",
    "                losses.append(loss)\n",
    "                update_cnt += 1\n",
    "                \n",
    "                # linearly decrease epsilon\n",
    "                self.epsilon = max(\n",
    "                    self.min_epsilon, self.epsilon - (\n",
    "                        self.max_epsilon - self.min_epsilon\n",
    "                    ) * self.epsilon_decay\n",
    "                )\n",
    "                epsilons.append(self.epsilon)\n",
    "                \n",
    "                # if hard update is needed\n",
    "                if update_cnt % self.target_update == 0:\n",
    "                    self._target_hard_update()\n",
    "\n",
    "            # plotting\n",
    "            if frame_idx % plotting_interval == 0:\n",
    "                self._plot(frame_idx, scores, losses, epsilons)\n",
    "                \n",
    "        self.env.close()\n",
    "                \n",
    "    def test(self):\n",
    "        \"\"\"Test the agent.\"\"\"\n",
    "        self.is_test = True\n",
    "        \n",
    "        state = self.env.reset()\n",
    "        done = False\n",
    "        score = 0\n",
    "        \n",
    "        frames = []\n",
    "        while not done:\n",
    "            frames.append(self.env.render(mode=\"rgb_array\"))\n",
    "            action = self.select_action(state)\n",
    "            next_state, reward, done = self.step(action)\n",
    "\n",
    "            state = next_state\n",
    "            score += reward\n",
    "        \n",
    "        print(\"score: \", score)\n",
    "        self.env.close()\n",
    "        \n",
    "        return frames\n",
    "\n",
    "    def _compute_dqn_loss(self, samples: Dict[str, np.ndarray]) -> torch.Tensor:\n",
    "        \"\"\"Return dqn loss.\"\"\"\n",
    "        device = self.device  # for shortening the following lines\n",
    "        state = torch.FloatTensor(samples[\"obs\"]).to(device)\n",
    "        next_state = torch.FloatTensor(samples[\"next_obs\"]).to(device)\n",
    "        action = torch.LongTensor(samples[\"acts\"].reshape(-1, 1)).to(device)\n",
    "        reward = torch.FloatTensor(samples[\"rews\"].reshape(-1, 1)).to(device)\n",
    "        done = torch.FloatTensor(samples[\"done\"].reshape(-1, 1)).to(device)\n",
    "\n",
    "        # G_t   = r + gamma * v(s_{t+1})  if state != Terminal\n",
    "        #       = r                       otherwise\n",
    "        curr_q_value = self.dqn(state).gather(1, action)\n",
    "        next_q_value = self.dqn_target(\n",
    "            next_state\n",
    "        ).max(dim=1, keepdim=True)[0].detach()\n",
    "        mask = 1 - done\n",
    "        target = (reward + self.gamma * next_q_value * mask).to(self.device)\n",
    "\n",
    "        # calculate element-wise dqn loss\n",
    "        elementwise_loss = (target - curr_q_value).pow(2)\n",
    "\n",
    "        return elementwise_loss\n",
    "    \n",
    "    def _target_hard_update(self):\n",
    "        \"\"\"Hard update: target <- local.\"\"\"\n",
    "        self.dqn_target.load_state_dict(self.dqn.state_dict())\n",
    "                \n",
    "    def _plot(\n",
    "        self, \n",
    "        frame_idx: int, \n",
    "        scores: List[float], \n",
    "        losses: List[float], \n",
    "        epsilons: List[float],\n",
    "    ):\n",
    "        \"\"\"Plot the training progresses.\"\"\"\n",
    "        clear_output(True)\n",
    "        plt.figure(figsize=(20, 5))\n",
    "        plt.subplot(131)\n",
    "        plt.title('frame %s. score: %s' % (frame_idx, np.mean(scores[-10:])))\n",
    "        plt.plot(scores)\n",
    "        plt.subplot(132)\n",
    "        plt.title('loss')\n",
    "        plt.plot(losses)\n",
    "        plt.subplot(133)\n",
    "        plt.title('epsilons')\n",
    "        plt.plot(epsilons)\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Environment\n",
    "\n",
    "You can see the [code](https://github.com/openai/gym/blob/master/gym/envs/classic_control/cartpole.py) and [configurations](https://github.com/openai/gym/blob/master/gym/envs/__init__.py#L53) of CartPole-v0 from OpenAI's repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# environment\n",
    "env_id = \"CartPole-v0\"\n",
    "env = gym.make(env_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set random seed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[777]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seed = 777\n",
    "\n",
    "def seed_torch(seed):\n",
    "    torch.manual_seed(seed)\n",
    "    if torch.backends.cudnn.enabled:\n",
    "        torch.backends.cudnn.benchmark = False\n",
    "        torch.backends.cudnn.deterministic = True\n",
    "\n",
    "np.random.seed(seed)\n",
    "random.seed(seed)\n",
    "seed_torch(seed)\n",
    "env.seed(seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n"
     ]
    }
   ],
   "source": [
    "# parameters\n",
    "num_frames = 10000\n",
    "memory_size = 1000\n",
    "batch_size = 32\n",
    "target_update = 200\n",
    "epsilon_decay = 1 / 2000\n",
    "\n",
    "# train\n",
    "agent = DQNAgent(env, memory_size, batch_size, target_update, epsilon_decay)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABIgAAAE/CAYAAAAt2/ipAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XmcZHddL/zP95zaeu+umZ59JlMJk0ASQgKTbl7igoAakCuK98HEBUQkl0e4rq+roD7KRVGf5169ymXRICGClyAXXKLkCoggqJDJhITs68wks3dnpnt6reWc833+OOdUnaqufT2n+/N+veaV6qrqqt90T7q7Pv1dRFVBRERERERERERblzHoAxARERERERER0WAxICIiIiIiIiIi2uIYEBERERERERERbXEMiIiIiIiIiIiItjgGREREREREREREWxwDIiIiIiIiIiKiLY4B0SYmIleJyAMisiwiPz/o8xAREYWZiJwQkdcM+hxERBQ9IvLrIvLn3uWDIqIiEhv0uYhawYBoc/tVAF9R1TFV/cCgD1NJRG4TkSdExBGRn65y+y+JyDkRWRKR20UkGbjtoIh8RUTWROTxyh/oO3nfKBORa0XkCyLyvIholdsPisjdIrLgfXw+GPzG1ehzUuXxXiMi3xKRVRE5JSJv6vJfiYiIiIgo9FT191T1Zwd9DqJOMCDa3C4D8EitG0XE7ONZqvk2gJ8D8K3KG0TkBwC8G8Cr4f49LgfwXwN3uRPA/QC2AfgNAJ8VkelO33eQxNXp/5MFAJ8B8LYat38YwByA3QCuB/A9cD8Hvpqfk0oicjWAT8H9GE4AeAmA+9o9OBEREREREQ0OA6JNSkT+GcD3AvigiKyIyJUicoeIfMSrIFkF8L0i8oMicr9XaXNSRN4beAy/NPKt3m0LIvIOEblRRB4UkUUR+WDF8/6MiDzm3fcLInJZrTOq6odU9csAslVufguAj6nqI6q6AOB3APy09xxXAngpgN9W1XVV/RyAhwD8aBfet9HH9ddE5LTXtveEiLzau970ykqf8W67T0T2e7d9h4jcKyKXvP9+R+Dxvioi7xeRfwOwBuByEZkQkY+JyFnvuX632TBPVZ9Q1Y+hdjCYAfAZVc2q6jkA/wjgmsD71/ucVPpNAH+mqv9HVS1VvaCqzzRzTiKiMBORpIj8sYic8f78sV+JKiLbReQfvO+BF0Xk6364X+t7BBERhY+I7BGRz4nIvIgcF28kh4i8V0Q+KyJ/5X09/5aIvCTwfrVeD7xXRP6yznPd5X3feFpE3h647b0i8hkR+YT3mI+IyOFGz0fUCwyINilVfRWArwN4l6qOquqT3k0/DuD9AMYA/CuAVQBvBjAJ4AcB/N8i8sMVDzcL4BCAHwPwx3ArRl4DN1h4k4h8DwCIyBsA/DqANwKY9p7/zjb/CtfArWbxfRvAThHZ5t12TFWXK26/pgvvW5OIXAXgXQBuVNUxAD8A4IR38y8DuAXA6wCMA/gZAGsikgbweQAfgFux9EcAPu+dxfdTAG6F+zl5FsAdACwALwBwA4DvB/Cz3hkOeC9KDjQ6bw1/DOBmERkWkb0AXgs3JGrHy70zPeSFWX/p/X2JiKLuN+B+jbsebnXkDNxQHAB+BcApuN/ndsL9vqcNvkcQEVGIeMH+38N9HbAXbufBL4rbiQAAbwDwvwGk4VbM/62IxDv4Wv9puN879gD4jwB+T0ReFbj9h7z7TAK4C8AHvXPyewv1FQOirefvVPXfVNXxqki+qqoPeW8/CDfQ+Z6K9/kd775fhBso3amqc6p6Gm4IdIN3v3cA+H1VfUxVLQC/B+D6elVEdYwCuBR42788VuU2//axLrxvPTaAJICrRSSuqicCFTM/C+A3vQoeVdVvq+oFuKHbU6r6Sa/K5k4AjwP4D4HHvcOrdrLgfhN6HYBfVNVVVZ0D8D8A3AwAqvqcqk6q6nNNnLear8ENw5bgfpM6CuBv23ysfXDDrR+FGyAOAfifbT4WEVGY/ASA93nf6+bhtin/lHdbAW6b7mWqWlDVr6uqov73CCIiCpcbAUyr6vtUNa+qxwB8FN7P3ADuU9XPqmoB7i94U3B/cdDy13qvq+AVAH7Ne031AIA/h/tLet+/qurdqmoD+CTcX06gnecj6gQDoq3nZPANEZkVd2DzvIhcghvybK94n/OBy+tV3h71Ll8G4E+8CpdFABcBCNxUvlUrcCtxfP7l5Sq3+bf7VUGdvG9Nqvo0gF8E8F4AcyLyaRHZ4928H0C1L9Z74FYFBT2L8o9J8HNyGYA4gLOBj+OfAdjR6HyNeL8p+UcAfw1gBO7neQrA/9vmQ64D+LiqPqmqK3ADwdd1ek4iohCo/Nr9rHcdAPw3AE8D+KKIHBORdwMNv0cQEVG4XAZgj//ztvcz96/DrQwFAj+fq6oDr/qnza/1ewBcrOhgqHw9cC5weQ1ASkRi/N5C/caAaOup3Gz1KbhljPtVdQLAn8INddpxEsB/8ipc/D9DqvrvbTzWIygl5/Aun/eqch6BO6tnrOL2R7rwvnWp6qdU9TvhflNRlMKVkwCuqPIuZ7z7Bh0AcDr4sIHLJwHkAGwPfAzHVbVhC1wT0t5zf1BVc97H4+NoP9R5EOVn37A1jYgooiq/dh/wroOqLqvqr6jq5XBbAn7ZnwdR53sEERGFy0kAxytet4ypqv9z8X7/jt4vWfeh9H2g1a/1ZwCkK15/VL4eqInfW6ifGBDRGNxEOysiM3BnFLXrTwG8R0SuAQBv2PL/VevOIpIQkRTcQCouIikpbfH6BIC3icjVIjIJd/bDHQDgzVN6AMBve+/zIwCuA/C5LrxvTSJylYi8yhtUmoVbQeN4N/85gN8RkUPius6bM3Q3gCtF5MdFJCYiPwbgagD/UO05VPUsgC8C+EMRGRcRQ0Su8Oc8NXFG8T6mCe/tlHdeqOrzAI7DnTMV8z42b4Eb9PjvX+9zUunjAN4qIpeLyDDczXFV/15ERBFzJ4DfFJFpEdkO4LcA/CUAiMjrReQFIiJwW5RtAE6D7xFERBQuRwAsewOgh8RdOHOtiNzo3f4yEXmjiMTgVvDkAHyzna/1qnoSwL8D+H3vZ+vr4G4crjrQOojfW6jfGBDRzwF4n4gsw/0B+DPtPpCq/g3cRPvTIrIE4GG4Q5Br+SLcL3LfAeA27/J3e4/1jwD+PwBfAfAc3DLM3w68780ADgNYAPAHAP6jNyeio/cVkZ8QkVrVREnv/s/DLQPdAeA93m1/BPdj90W4830+BmDIq9J5PdyhphcA/CqA13thTS1vhhvwPOqd8bNw5134Q6pX6gypvgzux9H/O6wDeCJw+xsB3ARgHm6LRAHALwVur/k5qfzYqOrtcMO4e+B+jHMAfr7O34uIKCp+F+6Mtgfhbrr8lncd4M5c+ye4LcvfAPBhVf0K6n+PICKiEPFm/bwe7jKC43C/dv85gAnvLn8Hd0HPAtwZdG/05hG1+7X+FgAH4VYT/Q3cjcr/1MT78XsL9ZW4cxWJiIiIiIiItjYReS+AF6jqTw76LET9xgoiIiIiIiIiIqItjgEREREREREREdEWxxYzIiIiIiIiIqItjhVERERERERERERbHAMiIiIiIiIiIqItLjboAwDA9u3b9eDBg4M+BhFRKN13333Pq+r0oM8xSPw+QURUHb9HuPh9goioula+T4QiIDp48CCOHj066GMQEYWSiDw76DMMGr9PEBFVx+8RLn6fICKqrpXvE2wxIyIiIiIiIiLa4hgQERERERERERFtcQyIiIiIiIiIiIi2OAZERERERERERERbHAMiIiIiIiIiIqItjgEREREREREREdEWx4CIiIiIiIj6QkRuF5E5EXm4xu0iIh8QkadF5EEReWm/z0hEtFU1DIhEZL+IfEVEHhWRR0TkF7zr0yLyJRF5yvvvlHc9v6gTEREREVE1dwC4qc7trwVwyPtzK4CP9OFMRESE5iqILAC/oqpXA3g5gHeKyNUA3g3gy6p6CMCXvbcBflEnIiIiIqIqVPVrAC7WucsbAHxCXd8EMCkiu/tzOiKirS3W6A6qehbAWe/ysog8BmAv3C/er/Tu9hcAvgrg1xD4og7gmyIyKSK7vcchirznV3L48mPn4Whnj2MI8JoX7cS20WTV21UVn3/oLJazFgBgNBnDD754NwxDqt5/YTWPLz16HrY2f7CXX74Nme0jZdc9dOoS9kymap4rChzH/dit5Ky699s1kcL3XrWj7Lr55RzOL2Vx7d6JsuufnlvBvSfq/Txb340H03jBjtG235/a841nLmBuOYs3XL930EchIqLm7AVwMvD2Ke+6nryW+NqT81gv2PiBa3b14uGJiCKlYUAUJCIHAdwA4B4AOwOhzzkAO73LTX1RF5Fb4VYY4cCBAy0em2hwbv/X4/jwV5/pymP9wquz+KXvu7LqbY+eXcK7PnV/2XX7poZww4Gpqvf/y28+iz/80pMtPf/3X70Tt735cNl1b779HtwycwC/etMLW3qsMHno9CX85zvvb3xHAPf/P9+HqZFE8e0/+fKT+OfH5vDv73l12f3e9w+P4mtPzrd9pj9444sZEA3AZ+87hW8eu8CAiIhoE+rG64mPfv0Y5pdzDIiIiNBCQCQiowA+B+AXVXVJpFTFoKoqIi3VU6jqbQBuA4DDhw93WItB1D8LawWkRxK4++e/q6PHedUffhWrdSpc/Mqh/3nLDRiKm/jZTxzF4nqh5v1XchYSMQNf+y/f29Tzv/0TR7GWt6s+jv/cUXXJ+zj92U+9DC/ZN1n1Pv/8+Bx+/W8ewrmlbFlAdGphHatVPi7ZvI2XHpjEh3/iZW2daXyopTyeiIhoqzoNYH/g7X3edRt04/XEzME0/uifnsTiWh6Tw4nG70BEtIk19YpFROJww6H/pap/7V193m8d8/qC57zrm/6iThRFKzkLE0Nx7JpIdfQ4yZiBvO3UvD1bcEOKPZMpjKfi7nPXCW6yBRtDcbPpcw0nTOSt8ud3HEXBVuSsjQFJlPjB1/6p4Zofj0M73WqeueUcXhSYbDC3lINTpX/QVsVQovmPLxEREbXlLgDvEpFPA5gFcKmXoypmMmmoAkdPLOA1V+9s/A5ERJtYM1vMBMDHADymqn8UuOkuAG/xLr8FwN8Frn+zt83s5ejxF3WiflvJFjCSNDt+nETM2BDQBGUL7m3JmImRpJvl1pupk7McJGPNzJ0PPH9FQFVw3LfrnSsK1vLux6ne52nHmDtjaW4pW3b93HIOVpWAyHIUptH8x5fCQaqP7CIiogERkTsBfAPAVSJySkTeJiLvEJF3eHe5G8AxAE8D+CiAn+vleV6yfxIJ08CRDuYMEhFtFs1UEL0CwE8BeEhEHvCu+3UAfwDgMyLyNgDPAniTd9vdAF4H94v6GoC3dvXERAO2mrMxmuy8XShu1g+I/CqeVNzEaCrmPXeDgCjeQkBU5fn9t+tVNkWB3yI2lKgXELmVQHPLueJ1lu3gwmoO8SpBkO04iNUYEE7hpi0Mbiciot5S1Vsa3K4A3tmn4yAVN3H9/kncc5wBERFRM1vM/hVArVdFr668ot9f1In6bTlnYe/kUMePk4gZyNUJYnJeBVEqbmAk4f6vWm82UM6ykYo1X9mUiBkoVFYQ2Vr23FG15gVp/setmqGEibFUDPOBgOjCah6qqLoJznYAg+UokcPPGBERNTKTSeMj//IMVnNWsWqbiGgrYr8EUYtWcgWMpTr/4SFhGijUazHzKoiSMROmIRhJmHVbzLKF1iqI4ubGFrNNV0EUrx+Y7RhLYm651GI2t+SGRbajG6pOWEFERES0Oc1k0rAdxbeeWxj0UYiIBooBEVGLVrJWV1rMmh1SnfJCn9FUrO6Q6pxlI9lqBZFVWUHkeI8V7YBoPW9hOGHCaBDo7BhL4fxSqYIoGBZVjiGyHYXJgCiS2GBGRET1vPSyKZiG4AjbzIhoi2NARNSi1ZzdlfLjZodUp7wqmJFkDCv5OgFRwSmGSc2oVkHkB0NRD4hW8zaG67SX+XaMV1QQBecROeUfAwZE0cSuQCIiamQ0GcO1eydwzzEGRES0tTEgImpBzrKRt53utJg1CIhylg3TEMRN93/TsWSjCiKnpQqiZJXn9yuIIr/FLGc1tWlux1gSc0u5YjvZ+cBGM7uihMhylC1mEcUZ1URE1MhsJo0HTi4WK7iJiLYiBkRELfADmm60mCWqVPAEZQsOUoG19aOpWIMZRHZLa+7jptSeQWRF+4ej1bzdcP4QAOwcTyFnOVjyPq/BCqLKgMhxtGHLGgEicruIzInIwzVuFxH5gIg8LSIPishLe3oejqkmIqImzBxMI287+PbJxUEfhYhoYBgQEbXAD2i6EhA1bDGzkQyEHKNNVRC1sOY+ZhS3lvk2ywyitXxzW0imx5IAgHmvzWxuqXZAxAqipt0B4KY6t78WwCHvz60APtKHMxEREdV148E0RMA5RES0pTEgImqBv2a+OzOIzIYziMoqiJLxuhVEOcsuzitqRtw0YDtaFoSUKoiiHhDZGE4002KWAoDioOr55dotZo6ygqgZqvo1APV+un4DgE+o65sAJkVkd0/PxDHVRETUwMRwHFftHMOREwyIiGjrYkBE1IJVL6DpxgyiuCl1K3UqA5/RZBNr7lusIAJKVUNAab191Nfcr+VsjDQ5pBoobS+r12LGCqKu2QvgZODtU951PcEh1URE1KzZTBr3PbtQ9rMREdFWwoCIqAXdbDFrvObeKW8x82YQaY2JuzmrvCWtkYQ3/DoYUvmVQ7lCtH8wWs1bGG5ySDXgtpY5jmJ+OYf0SAKAGwgF2Ta3mPWbiNwqIkdF5Oj8/Pygj0NERJvcTGYb1vI2HjmzNOijEBENBAMiohYUA6JubDEzjbq/ocpZ5UOnR5Nx2I4iWyW8UdW2ZhAB5RVE/kyiyFcQNdliNpqMYShuYm45h4W1PCxHsXvCbTurrCCyVWGyHKUbTgPYH3h7n3fdBqp6m6oeVtXD09PTbT8ht5gREVEzbsxMAQCOHL8w4JMQEQ0GAyKiFvgziMb6NKQ6FS/fYgYAy7nChvsWbIUqWp5BBJTPG8rb7vYy21FYEQ6JVnNWUy1mIoKd40nMLeeKc4h2TwwBqN5iZpoMiLrgLgBv9raZvRzAJVU926snY6ZHRETN2jGWwuXbRziomoi2rM5f5RJtIX4FUXeGVNcPiHKWU2x3Akqh1ErWwo6x8vtmvbX0LVUQmVUqiKzAwGrbQcyMXoZsO2411XATARHg/jA4t5QtziHaM+lVEOnGNfesIGpMRO4E8EoA20XkFIDfBhAHAFX9UwB3A3gdgKcBrAF4a6/PxAIiIiJq1kwmjbsfOgvH4XIKItp6GBARtWA1Z0EETbUvNZIwTViO1vwBJFuwkYqVnscPpVZz9ob7+jOD2mkxC4ZUObt8HtFwYsO7hd5a3g/xmvscTY8n8eiZpeKA6moVRKrKIdVNUtVbGtyuAN7Zp+MA4OeMiIiaN5NJ49P3nsQT55fxot3jgz4OEVFfRa88gGiAlrMWRpMxSBcqSYoBTY1WrmzBKW8xS9ZuMcv5FUTttJiVVRBtHFgdNWt592PRfAVREnNLWcwXA6KNM4j8i6bBL5lERESb2ezl2wAA9xzjHCIi2nr4aoeoBSs5qyvzh4BSQFRr1X22YCMZqCAaS5VazCr5j9FKBVGySgVRMCyqda6wW/XaAJut8toxlsJq3sbx51cxlooVK7WCAZF/OYIddwQOqSYioubtnRzC3skhHDnBOUREtPXw5Q5RC1ayVlfmDwHVW7yCclb1CiJ/DlJQtuDPIGq9gsjfXAaUVxBFNSAqVRA1GxC5q+4fObOEHWPJYghkVQ2I+CUzajg2ioiIWjWbSePI8YtQ/oaBiLYYvtohasFq3urKinsASFZp8Qpyt5iVQg7/easFRMUKonhnM4jydvRbzFZbHCS+c9xtKXvq/DJ2jqeKIVBZBZGygija+AM+ERE1byaTxvMreRx7fnXQRyEi6iu+3CFqgT+DqBviMbe0oVoQo+pu4grOFKpXQeQPqU61VEHkPn9wi1l5i9nGYdhRsFZosYJo3K0gshx1K4i8kpOygMhmBVFUsYCIiIhaNZNJAwDX3RPRlsNXO0QtWMlZxVlAnUqYboBRqFJBVG2mUDJmIGZIjRlE/pDq1iuIcjUGU0e1gmjN2/LWbAWR32IGADvGUzCNjQGR5bgfC24xIyIi2vwy20ewfTTJgIiIthwGREQtWOliBVG9GUTFiqBABZGIYDQVqzGDqI0198UZRIEtZjWqiaJk1VtzP9TkRreJoXjxc7FjLImYWaWCyGsxMxgQRRJHSBARUStEpDiHiIhoK2FARNSClVz3h1RXGwad9SqCUhUVQaPJWP0KohZazKrOIAoOqS5EMyBaa3EGkYhgetStIpoeS8LwWsz8qiGgFBaxgih6OKSaiIjaMZNJ4/TiOk4trA36KEREfcOAiKhJjqNYzXdxzb1Zu4LI30pWOVNoNFm9gsgPmSoDpXriVSuISqUW0a0gam0GEQDs9OYQ7RxPFUMgR6tsMWPaEEksICIiolZxDhERbUUMiIiatFawoYqubTErVvBUCWKyVVrMgDoBURtr7qs9f95yigFJVGcQredtmIa01G63Yyzl/TdZnEFk2dXW3DMgihrhmGoiImrDVTvHMJ6KMSAioi2l4SsoEbldROZE5OHAdX8lIg94f06IyAPe9QdFZD1w25/28vBE/eS3do0m4115vGSdGUTZYuBT0WJWYwZRO2vu41UqmPK2UwzAorrFbDVvYThhQlqo9vE3mQWHVAcriCy/xcxk2EBERLQVGIZghnOIiGiLaaYU4g4AHwTwCf8KVf0x/7KI/CGAS4H7P6Oq13frgERhsZIrAABGks1X6dRTd0i1VbuC6LkLG3vhq209ayRZo4JoNBnD4lohshVEazm7pfYyAHjVC3fg0noBo8lYsYLKCgypdrzLBlvMIkk5pZqIiNowm9mGf3psDnNLWewYTw36OEREPdfw1aSqfg1A1ehc3F/RvwnAnV0+F1HoLHsVRN1bc+8HNBsrdYoziCoqgsZSMSzXaDETKT1mM4oziKzSi+eC7RS3tFUbnh0Fq3kLI4nWPkevvGoH/uTmGwCUNpWVr7nnkOqoYqZHRETtKs4hOsEqIiLaGjqdQfRdAM6r6lOB6zIicr+I/IuIfFeHj08UGqs5N7TpVotZvIkWs2oVRKvV1txbDpIxo6W2KtMQmIaUBVR+BREQ3YBoLW9juIMqr1iVgMi/zDX3REREW8c1e8YxnDDZZkZEW0anAdEtKK8eOgvggKreAOCXAXxKRMarvaOI3CoiR0Xk6Pz8fIfHIOo9v8VstB9bzGq0jI0kY1jL22XhBeBWELUyoNoXN6Vsc1nBdorr4ZtpMbu4msc7P/Wtqm1vg7KaszDcYgVRUGnN/caAiBVE0cQGMyIiakfMNPCyy6YYEBHRltF2QCQiMQBvBPBX/nWqmlPVC97l+wA8A+DKau+vqrep6mFVPTw9Pd3uMYj6pustZsUZQBtfvubqVBAB2DCoOmc5La24L57BNMqCoJxXiZQwjaYqiL7+1Dw+/+BZ/Nyn7itWPQ3aesHGSIsziIL8QdROlRYzVhBFDz9jRETUidlMGo+fW8biWn7QRyEi6rlOKoheA+BxVT3lXyEi0yJiepcvB3AIwLHOjkgUDn4oM9KlCqK6W8xqbCXzw6lqAVE7FUSJmFE2pLpgO4jHDPf6JgKiJ84tQwR4+PQS3v/5x1p+/l5YzVkY7uBzZFYbUq2sIIoyzqgmIqJ2zWS2AQDuPbEw4JMQEfVeM2vu7wTwDQBXicgpEXmbd9PN2Dic+rsBPOitvf8sgHeoKmsyaVNYLQZEXdpiVqfFrHYFkTv/aCVbHhBlC3ZLG8yCZ6hcc580vYCoyvDsSk+eX8ahHaN4+3dl8MlvPou///aZls/QbWt5G8Px9j9HplRZc+9VeZkMiCKnlblcREREla7bN4FEzMCR4xcGfRQiop5r+Gt2Vb2lxvU/XeW6zwH4XOfHIgqf5ZyFRMxoq1KnGsMQxCqGRPuKQ6ornmu0XgVRGy1m8ZiBQrCCyFLETQPJmIFcoXEF0ePnlnHDgSn86k0vxH3PLuA9f/0QXvGC7UiPJFo+S7es5qyOqrxihvtxtOyNFUQmwwYiIqItJRU3cf3+Sc4hIqItodMh1URbxkrWwliX2st8tVq5cpYDEXeIdNCoV720MSCyN4RJTT1/lQqihN9iZtcPiFZyFk4trOOqnaOImwbe9aoXYCVn4cSF1ZbP0U1reRvDHcwg8vKh6mvuTQZEUaTsMSMiog7MZtJ4+MzShp+/iIg2GwZERE1ayVnFCp5uqRUQZQtu4FPZHlOrxSxXaLOCqDIgshzETWNDcFTNk+eXAQBX7XIXFU4MuVVDS+uFls/RLXnLgeVoVyqIbA1uMXM/FgYriIiIiLacmUwatqP41rOcQ0REmxsDIqImrWQtjHSwPr2ahFm9UidbqL6VrNRiVh7CZK321txXVgr5FUTJeOMtZk+e8wKinWMAgHHvbEvZwf12bS3vPncnFUT+nCG7bM29+18/PKJoYf0QERF14qUHpmAawjYzItr0+GqHqEm9qCCK11gnny3YGwZUA8E19+Vzi3IFp+Mh1aqKvOUgYUpTFUSPn1vGcMLEvqkhAMD4kFvdtJwdXAXRat79uHQ/IHLKbqPoYNEXERF1aiQZw4v3TuAeDqomok2OARFRk1Zy3Z9BlKwzg6ha4FMMiCpbzCynaqDUSCIwpNqfs5Nocs39E+eWcWjnGAwvNBnzK4jWB1hBlPMriNr/PPkZkFWlgogBERER0dY0m0nj2ycvFReJEBFtRgyIiJrUqxlEhaotZtUriExDMBQ3N7aYtbnmPm5KscXMD4TcLWYmclb9H4CePL+MF3rtZQAwFDcRMyQUFUQjyfYriEQEpiHFqiEAsFhBFG3sMSMiog7NZNLI2w4eOLk46KMQEfUMAyKiJq1kO1ufXk3NIdWWg2SNiqDRVKz6mvt2WsxiBgqW++rZD6r8CqJ6M4jml3O4sJrHlbtKAZGIYHwojqUBBkSlGURb+h+LAAAgAElEQVSdfZ7cgKj0dnHNPQOiyBHwc0ZERJ07fFkaIuAcIiLa1BgQETVpuQctZrWHVNtI1Qh8xpIxLG9oMatecdRIPPD8wQqiRmvu/Q1mLwwERIDbZjbYFjOvgqjDgChWWUFka/F6ih4WEBERUacmhuN44a5xBkREtKkxICJqQt5ykLec4gygbqlVQZSr0WIGuBVEq4EKIlXtqILIf/58oIIoGTOQK9QOiB73N5hVBETjqfiAW8zcj8tQB0OqAcCU8goif2A1K4iih0OqiYioW2Yzadz37ELV8QBERJsBAyKiJviBTC9mELUypBpwB1UHW8zytgNV1GxJq/v8VSqIEqYbENWtIDq3jG0jCWwfTZZdPz4U63jN/dETF3H/cwttve9aF2YQAYBpllcQ2WwxIyIi2vJmMmmsF2w8fPrSoI9CRNQTDIiImuAHMl2vIGpxzT3grloNtpj579/2DCIvCCrYgS1mDdbcP35+eUP1EACMJeNYWu+sguh3Pv8Yfv/ux5u+/5nF9WKFz2oXtpgBbgVR+RYzBkRRpsomMyIi6tyNB9MAOIeIiDYvBkRETfADmZ60mFWdQeQgFa89gyhYQeS3grVTQRQPBEFlW8zitbeYOY7iqfPLuHLnxoBofGjjfKRWXVjJYW4529R9F9fyeOV//yo+deQ5AKUKouFOW8wMKQ6mBgIBEfuVIoefMSIi6pbpsSQunx5hQEREmxYDIqIm+LNt+tVilq0zdLpyi5kf5HRaQRScQVSvguj04jrW8nbVCqLxVOdbzBbXCphfzjV138fOLiNvOcUf1NbyNhIxA3Gzsy9tMUOKg6mBQEBkMm4gIiLaymYzaRw5cbH4swER0WbCgIioCSu9qiCqEcTkCvVnEK3mrGLbTLbQfotZ3DRQsBWOo4EKIkEiZsBRwKpS3fTcxTUAwMFtIxtuG0vFsZa32x7emLccrOQsrObtskHctTxxbgkA8NCpRQDumvtOq4cAwDCkOHcIQLHdjFvMook/whMRhYuI3CQiT4jI0yLy7iq3HxCRr4jI/SLyoIi8bhDnrGYmk8Zy1sIT3sIOIqLNhAERUROWvbBirAcVRJVhiqo2rCAq2FqcPVSqIGo9GPFDpYLjFM+R9LaYuY+9Meg5vbAOANg7ObThtvEh9+Oz0mab2eJ6vni5mSqiJ86vAABOXFjDpbUCVnN2xyvuAX/N/cYKIoMtZpHDTxkRUbiIiAngQwBeC+BqALeIyNUVd/tNAJ9R1RsA3Azgw/09ZW2zmW0AgHuOXxjwSYiIuo8BEVET/MBjpA8VRP5WspoBkXcGv83MD3FqzSyqJ+61TOUtp2wGUcILiKpVN51eXIcIsGsiteG28VQcANpuM1tYLb3f/EoTAdG5peLf+8HTi12tIKo2pJoVRNHEGdVERKEyA+BpVT2mqnkAnwbwhor7KIBx7/IEgDN9PF9deyaHsG9qiHOIiGhTYkBE1ISVnBtc9GNIdaOWMf8M/jDo4pDqNiqIEt6snoKtxQqiRCwQEFVpFTu9uI4dY8nifYL8Cqul9fYqiBbWmq8gUlU8eX4FN12zCwDw4KlLWM3bGO7C5yhmCBxuMdsUhCVERERhsxfAycDbp7zrgt4L4CdF5BSAuwH85/4crTkzmTSOHL/ILZlEtOkwICJqwnLWggi60r4U5LaYaVkY4beM1aogmhpOACiFKVm/xaydCqJApZAfBsVNoxg2+eFT0JnF9artZQAwPuRWEC23WUG02EJAdHpxHSs5Czdm0ji4bRgPnlrEet7CSDcqiKqsuTeEYQMREVGf3ALgDlXdB+B1AD4pIht+0BGRW0XkqIgcnZ+f79vhZjNpXFjN45n51b49JxFRPzAgIgrIWw7u+vaZDb8RWs5aGE3GYHS5gqRapU6uQQVResQLiFbzZfdPdVRBVGoxSwRbzOyNq+5PL65jT42AqFhB1GZAdDHYYtYgIHryvDsc8qqdY7hu36RbQZSzu9JiFjPLK4gsRxEz+OUyqpRjqomIwuQ0gP2Bt/d51wW9DcBnAEBVvwEgBWB75QOp6m2qelhVD09PT/fouBvNeHOI2GZGRJsNX/EQBXz9qXn8/J3345EzS2XXL2ULxfk63eQHNMGAKFuoX0HkB0QX/ICogwqiRGAYdT4wpNo/V+WQasdRnF3MYu9UjQqi4gyizlrMxlOxhgHR4972kCt3jeG6fRM4eymLkwtrGO5ClZdpGGUVRI4qmA81L0zbaVjzRUQUOvcCOCQiGRFJwB1CfVfFfZ4D8GoAEJEXwQ2I+lci1MDBbcOYHkviCAdVE9Emw5c8RAH+/J/gLBzAnanT7Q1mQKlKKDgM2j9Do4DoYkUFUTtr7oMVRIXAkGo/bKoMiJ5fySFvOw1bzJbW228xS8UN7E8PNxxS/eS5ZeyZSGE8Fcd1+yYBuJVeI8nOK4hMcUMhn2WzgqhZYdxOwxERREThoaoWgHcB+AKAx+B+P3hERN4nIj/k3e1XALxdRL4N4E4AP60hGvgjIpjJpHEP5xAR0SbT/Ve8RBFmOW4gUjlkeblXFURVAqLSDKLqgcRwwkQyZpRazDpYc5+oMoMoETOQNDeeC3Dby4DqK+6B0gDtZiqIPnvfKTxwcgG/+8MvLl53cbWA9HAC02PJpiqIrto1BgC4du84DAEcRVcqiGKGAcuuqCBiKUqzittpAEBE/O00jwbu07/tNPy8ERGFjqreDXf4dPC63wpcfhTAK/p9rlbMZtL4/INncWphHfvTw4M+DhFRV/BX4kQBfihQOUNnOdubCqJ4lSAm22ArmYggPZIItJh1suY+UEHk/d3rrbk/s5gFgJoziExDMJaMNTWk+t+feR6fu+902W/eFtfymBxOYHo0ibnlbM33LdgOnplfwZVeQDSciOHQDvdyV4ZUG6XNZYAbHMZMfrlsUuS30xARETUyk0kD4BwiItpcGr7iEZHbRWRORB4OXPdeETktIg94f14XuO093tyJJ0TkB3p1cKJe8EOByhap5VyhJwGRH8QUqs4gqv2/Z3okUWoxszpYcx8IgnLFFjMpm00UdHpxDQBqziAC3DazZtbc245ivWCXVRstrOUxNRLHjvEknl/Jlw2KDjrx/CoKtuKFXkAEAC/eNwEAXVpzb8DW8i1mXHHfVX3dTsPifyIi6rYrd4xhYijOgIiINpVmfiV+B4Cbqlz/P1T1eu/P3QDgzZm4GcA13vt82JtHQRQJ/mDi6hVEvRtSHQxisg3W3APlAVG2YEPEDXZaFQ8MyS7YDhKmAREphk3VKojGkrG67XZjqVhTW8z8aq1zl0qVQotrhWIFke3ohllQvuKA6p2lgOglfkDUhQoi09i45t7kivtmhWo7jbDHjIiIesAwBDceTOPICQZERLR5NAyIVPVrAJr9yvcGAJ9W1ZyqHgfwNNx5FESRYFeZQaSqWM5aGB/qXQVR+RazxmvrKyuIkjE32GlVcEh23nKK56m15v7Uwnrd6iHA3WTWTIuZP+/p3FIpILq4lvdmEKUAoOag6ifPL8M0BFdMjxav8wdVd6PSyzQ2rrlnBVHTIr+dhoiIqBmzmTSOP7+KuaXabfFERFHSyVCNd3nriW8XkSnvumZmTxCFVqHKDKK1vA3b0d5UELUxpBqoCIgKdt1qo3pKM4gUBdspViEVW8wKlRVE6zXnD/nGh2JNt5gBwLlL68W3L60XMDUcx/RYEgBqDqp+/NwyDm4bLvt7X7dvAn/8Y9fj+6/e1fC5G6msIHIYEDUtlNtp2GNGREQ9MHu5O4foHraZEdEm0W5A9BEAVwC4HsBZAH/Y6gN0a7YEUTdVm0G07M3I6fea+3ozhbaNJLCSs5Cz7GIFUTuClULBCqJklcomwN1itmcyVfcxx1LxsoBtKVvAN565sOF+VjEgckOgpfUCVOG2mDUIiJ48v4wX7hovu05E8MM37MVIF2YQmSLFajL/rDEGRE1T1btV9UpVvUJV3+9d91uqepd3+VFVfYWqvsRrU/5ir87CzkAiIuqVq3ePYyRhcg4REW0abb2qVNXzqmqrqgPgoyi1kTUze8J/jI5nSxB1mx9aXCoLiNzLvZlBtHHWjz+kOlmngmhqJAEAWFgtIFuw2xpQDZTmFhUsRd52ihVF1SqbVnIWLq0XsHey/irX8VSsGKoBwMf/9QR+6mP3FCujfMUKoiW3guiiN28oPVI/IMoWbDx3cQ2Hdo5uuK1bTFPKtphxSHW0KUuIiIioB2KmgZcdTDMgIqJNo62ASER2B978EQD+hrO7ANwsIkkRyQA4BOBIZ0ck6h/Lq5gJbtbyq2HGe7jFLFipk/OGTterCtrmBUQXVnPIWU5bK+6Dz5+zK2YQVRmefWbRDXKaqSBazhaKM3yemluG5Wixfc9XOaR60QuIJofjGEmYGIqbVQOi48+vQhVl84e6LWYwINos+FkjIqJems2k8cT5ZSysVl+sQUQUJQ1f8YrInQBeCWC7iJwC8NsAXiki18Od7HACwH8CAG/OxGcAPArAAvBOVbWrPS5RGFlVWsyWii1m/ZpB1HjodHrErbBZWC1492+vgsgPggpWaYsZUAqnggHRaS8g2tdoSPVQDI4Cq3l389ux+VUAgF0ZEHktXGe9gGhh1f2YTw0nICKYHktWHVJ9/Hn38TLbR1r4m7bGFOGaeyIiImpoJuPOIbr3xEV8/zWdz0EkIhqkhgGRqt5S5eqP1bn/+wG8v5NDEQ2KXWXNvd8u1YsKIr/Fq7LFrFHgkx5xwyq3gsjuwgyi8goiEUHCNMrOdXrBryBqvMUMcD9uo8lYMdCxnPJ5Rn4Yd97b/LEQaDEDgOmxJOaWNgZEx+ZXAPQ4IDKkLNCylQFRlPVw/DUREW1x1+2bQCJm4MhxBkREFH2dbDEj2nT80CJbcIozc3o6gyjQ4uXLFhq3jPkVRBdX88gWnLrziuqJl1UQafFt/2zBuUFnFtcRMwQ7xhq3mAFuyHZuKYt1b6ZScCsYUArjFtbcOUoLgRYzAJgerV5BdGx+FbvGU10ZRl1L5RYzVhBFF4dUExFRLyVjJm7YP4kjJziHiIiijwERUYAVCGr8yqFiBdFQD7aYeUOqC8EKIqvx2vqJoTgMARZW88hZNlJttpjFDIFIoIIoEBAlYxUVRIvr2DWRahiU+B+n5axVbC8DNgZEVqBC59ylLBbWCogZglEv+Nkxnqw6g+jY86u4fLp31UOAGxA5gbITy+YWsyhjAREREfXSbCaNh09fwkrOanxnIqIQY0BEFBAMMfw5REvrBZiGYKhBaNOO6kOqnYaBj2kIJocTuLCaR66DCiIRQdw03IDIdhCPlVcQ5SuGVO9t0F4GlFrMltYLOPZ8KSCqnEFkO4oxr23v3FIWi2t5TI0kirOXpkeTuLReKKtiUlUcm1/peUAUq6wgUoXBUpRIEo6pJiKiHpvJbIOjwH3PLgz6KEREHWFARBQQ3Fy1FKggGkvF6g6Nble1IdVZy24q8EmPJNwWM6v9NfcAkPRmDVVWELktZuUziJoJiPzQZylbKM4LAoDChhlEDvZNDQNwK4guruYxNVxq4/NX3T+/UtoKcnE1j6Wshcz23m0wAwCjyhazmMmggYiIiDZ66WWTiBmCe45dGPRRiIg6woCIKCA4SNmvIFrOFoqhR7eZhsA0ZMOQ6mZaxvyAKFdw2h5SDQDxmIGC7W0xi5VCkGCLmWU7OLeUxd4GG8wAYHyoNKT6eLCCqLLFzNFi4HRuyW0xmxxOFG/3A6Jgm5lfkdSPCqLKgIgVRNGlnFJNREQ9NJyI4dq9EzhynHOIiCjaGBARBQTn4vibzJazVrFtqhcSXouXr9mh0+lhLyCynIYzixo+v+W2mFVWEPnnOr+cg6PA7okWKojWCzg2v1oMr6zKNfe2YmIojrFkDOcueS1mVSqIygIiryLpikFUEHEGUSQx1yMion6YvTyNb59aRLZgN74zEVFIMSAiCrAdLYYkS+tui9lSDyuIgI2zfpoNfNKjfkDU/pp7AIjHBAVbUbCc8i1mZmmL2blL7or73ZP1N5gB7jaPZMzA8yt5nFpYwwt2uGFO5Zp7P3TZOZEqDqn2V9wDtSuIEqbRVCVTJyoriCxHYRr8chlVrB8iIqJem82kUbAV9z+3OOijEBG1ja94iAIsRzE1UlrTDvgziHpYQVQx6ydXaLzFDAC2jSSwsOZWEHUSEJVVEMWCW8zMYnB17pIb0uwabxwQAW6b2cOnL8FR4MqdYwCqbDFzFKYp2D2RwtmlLBZW82UtZttG3IBobjlbvO7Y/Cou2zbc85XzpmGUnddxFCa/WkYSC4iIiKgfXnZZGiJgmxkRRRpf8hAFWI6DsVQcMUNwaT0YEPWwgsg0NswgaibwmRpOwFFAFUh20GJW3GJWWUEUqGw661cQTTQXEI2lYnj4zCUAwKGdbgVR5Qwi23EQMwS7xlM4NrfihnOBFrNEzMDUcHxDi1lme2/nDwGA6fUlOd6ZLcdBjBVEREREVMPEUBwv2jWOIyc4qJqIoouveIgCLNttexofipfW3GcLvZ1BFKuYQWQ5SDUxg2jbaKnappMKIn8Ydd4ur0QKVjadX8oiGTMwMdTcx2E8FUe24L7voR1uBVHBrtxipogZBnZNpLCcc9v5pgIVRACQ2T6Ce45fhOMoLNvBcxfXcPl0b+cPAShuLPOriBx15xJRNHFGNRER9cNMJo37nl0o+8UfEVGUMCAiCvDXmU8MxbGUteA4ipWchfEeVxAV2txi5uu0gsjdYqZlFUTJsgqiLHZPpCBNTvz1N5lNjyWLVUEbtpjZ7sd6V6AqqTIg+smXX4an51bwL0/O49TCOgq29nyDGYDixjJHgxVEDIgiiVOqiYioT2YzaWQLTrGKmogoahgQEQX4w4jHUzEsrRewkregip7PIApWEDU7pDoYpnQ0gyhmYL1guwO661QQ7WqyvQwobTLLbB8pzguqnEFkOwrTazHz+fOffK+/bg92jafw0a8fw3F/xX0fWsxiFWe2be353CMiIiKKthszaQCcQ0RE0cWAiCjAchzE/RazbAHLWbf1qV9bzAq2A9vRpgKfYItZJ2vu46aBtZxdvOxLBgKis5eyTQ+oBlBsybtieqQ4u2fDmnt/BlGdCqJEzMBbX3EQ//7MBfz9t88AQF9azPwwyPbObKsW5xIRERERVbN9NIkrpkcYEBFRZDEgIgqwvEqR8ZQ7g2jZ22TW0wqiwJDqbMENavpdQbTizQDauMXMhuMo5pZy2DXR/Gr58SE3ULt8+2hxno8dWHPvOApH3SBmd+BxKwMiALh55gBGEib++v7TmByOl7XW9UoxIPJazGxv4xpFDz9rRETUTzOZbbj3xMUNrfVERFHAgIgowJ9BND4Uw1LWKlYQ+YFHLyRiBnK2HxC5/21mSHUqbmIk4QZJna65X817AVEgBPFb3y6u5ZG3HewaTzb9mH4FUWb7yIZ2LaAUvMQMwdRwHImYAZHS7KKgiaE4bp45UHy8fii1xbmfD9thBVHUKSdVExFRH8xm0ljOWnjs7NKgj0JE1DIGREQBheIMIreCyN9k1vMZRBUVRM0OnU57bWbJJoZa13v+1SoVRAnTbTE7dykLAC1VEE16g6mv2DFaqsYJBER+u1nMNCDiziGaHIrXnPPz1lcchGkIruhDexlQCoj8oifL4QyiqGKuR0RE/TTDOUREFGG9K4sgiiA7MIMoZzm4sJIH0I8ZRG4wdH7JDWN2jDVXrZMeSeLkxfWmKo5qiZuCghfYBGcQJWIGVIFTC+sA0NKQ6v/wkj2YGIojs30EJy+uAUDxOYBSZY5fXbRrPFV3S9i+qWH8+VsO92VANVC9gohbzIiIiKiRPZND2J8ewpHjF/Ez35kZ9HGIiFrCgIgooDSDyP1f49SiG470MiBKmqUtZn4Ys29quKn3TXuVOp1WEFW77LetPXfR3R62u4WAaDwVx+uv2wMgMM8nMIPIrybyb7t5Zj8W1gp1H/N7r9rR9PN3KlZR9WSzgijyVFlNRERE/TFzcBu+8sQcVBXCbz5EFCFsMSMKKM0gcoOX015gM97DFrN4YEi1X22zb6q5dq70iFtplOyogsioetkPi569sAbTEGwfbX4GUVC1GUT+Zf+2N750H94Wot+yVbbFMSCKLuGYaiIi6rPZTBoXV/N4Zn5l0EchImoJAyKiAMtRxAyjFBAtriFhGh2tkW8kETOK7VenFtaxfTTZ9PNtK84g6myLWb3Lz11cw46xZNsBSczcuOa+VEEUzi9BGwIiZUAUdRxRTURE/eLPIbqHc4iIKGLC+eqMaEAsx0HMW3MPuIFNL9vLgPIh1acW17A/3fwwaH8tfCcBVsI0ql7229aevbDW0vyhSmYTFURh428ss1XhOApVMCCKKFb2ExFRv122bRg7xpIcVE1EkcMZREQBtjeDaMJba3/uUrbpdq92lQVEC+u4bt9k0+/7+ut2I2852DaSaP/5q7SVBS+fXlzHNXvG2378WJUZRJY3cylmhvPVezHUsjX0YRYRERGFi4hgJpPGPccucg4REUUKK4iIAgr+DCKvgshytNhu1isJb0i17SjOLK63FEjtTw/jF15zqKMfPOKxGjOIvMu2ox1VEPkhUPkWs/Ih1WHjn9l2FI66ZzVCelZqjiqbzIiIqH9mM2mcW8oWF5AQEUUBAyKiALtiBhHQ2w1mQKlS59TCGgq29rxiacPz12oxCwy+3jXeQUBklIImn12sygnnlyAj0GLGCqJo42eNiIgGYSazDQDnEBFRtDR8dSYit4vInIg8HLjuv4nI4yLyoIj8jYhMetcfFJF1EXnA+/OnvTw8UbdZtgPTECRjRjEsGUv2toLIHzDtb7podsV9t8TL2spKL6eTgbCokwoiP1cpm0Fkh7yCKBBq+WGWwfLwSGP9EBER9dOhHaOYHI7jyPELgz4KEVHTmvn1/R0Abqq47ksArlXV6wA8CeA9gdueUdXrvT/v6M4xifrDrSASiAjGvTlE/aogOja/CqD5FffdkiyrICoNuw7OI9o90f6ZRAQxQ4pzh4BgBVE4Qxe/sCkYEIX1rFQfcz0iIhoEwxDceDDNCiIiipSGAZGqfg3AxYrrvqiqlvfmNwHs68HZiPrOnUHk/m/hzyEaS/V+BhEAPOMFRHsn+xsQxQNVQ8HL/hYzoLMWM8Cd6WOXbTFzwyIzpEOqq1UQmWY42+GIiIgonGYzaTx7YQ3nLmUHfRQioqZ04xXPzwD4P4G3MyJyv4j8i4h8V613EpFbReSoiBydn5/vwjGIOudXEAHAmDeHyK8k6hV/MPSx+RXsGEt2tLK+HWVVQzU2mu0YT3b0HDHDqLrmPh7SGUSmURpSXQyIWIoSaZxRTURE/TbrzSE6coJVREQUDR29OhOR3wBgAfhf3lVnARxQ1RsA/DKAT4lI1f3Yqnqbqh5W1cPT09OdHIOoK1TdMMAPByaG+lRB5LeYPb/a9/YyAIibwQqijQFReiTRcWhlVrSYhX0GUTAg8qud2GIWTVwtTEREg/Ki3WMYTcY4h4iIIqPtgEhEfhrA6wH8hHr7g1U1p6oXvMv3AXgGwJVdOCdRz1VuqxpP9XcG0fxyru8DqoPPD1RsMfOu77S9DHBDKKvaFrPQtpi557IchZcPcc09ERERtSRmGnjZZVM4wjlERBQRbQVEInITgF8F8EOquha4flpETO/y5QAOATjWjYMS9VoptPBmEPktZn0KiID+D6gGaq+598/VyQYzn2nUmEEU0tCluOaeFUSbhnKPGRERDcBMJo0nz6/g4mp+0EchImqomTX3dwL4BoCrROSUiLwNwAcBjAH4UsU6++8G8KCIPADgswDeoaqMzCkSNlYQxcv+2yvBLWKDrCCKGVJWJdPNgChmGCjYVSqIQhq6+JVNtqNwNNztcERERBRes5k0AOBeziEioghoWBqhqrdUufpjNe77OQCf6/RQRINgV8zFKa25788MIgDYnx5ABZH3/MFzAMBQ3MRQ3ERm20jHz+FuMQvMIHLCHboUZxCphv6s1BwOqSYiokF48b4JJGMGjhy/iB+4Ztegj0NEVFdve2eIIqTgBRj+0OaJPm0xK28x638Fkb9FLV6xxj1uGviHn/9O7J3sPLQyjfIZRP6Q6srnDAuz2GLmhH6gNtXHGdVERDRIyZiJGw5Mcg4REUVCOF+dEQ1AcZ25t3r9tdfuxm+87kU4kO5taBMMiPZMdt7O1e7zV1YQAcAV06MdbzAD3FayKM0g8s9l2YEWMyYNTRORm0TkCRF5WkTeXeM+bxKRR0XkERH5VL/PSERE1C8zmW145MwlLGcLgz4KEVFdDIiIPJUziNIjCbz9uy/v+ZpsfzD0zvEkkrHOw5h2nz/Rw2qeqM0g8gMiJ9hiFtKNa2HjLSr4EIDXArgawC0icnXFfQ4BeA+AV6jqNQB+se8HJSIi6pPZTBqOAvc9uzDooxAR1cWAiMhj2d62qj4HAX7lziDay4LPX62CqFuiNoOofM19uMOsEJoB8LSqHlPVPIBPA3hDxX3eDuBDqroAAKo616vDCPh5IyIKm61WaXrDgUnEDME9bDMjopDjDCIiz6BCC79yZxAr7oHgDKLe/b0rZxCVKojCmVEXK4icQAURW8yatRfAycDbpwDMVtznSgAQkX8DYAJ4r6r+Y+UDicitAG4FgAMHDnR0KA6pJiIKh0Cl6ffB/R5xr4jcpaqPBu4TrDRdEJEdgzltdwwnYnjxvgnOISKi0AvnqzOiARhUaOFX7uzfxBVEccMoDnsGwl9BZAYqiOyQnzWiYgAOAXglgFsAfFREJivvpKq3qephVT08PT3d1hMx1yMiCp1QVZr2y2xmGx48tYj1vD3ooxAR1cSAiMgzqG1V46k4fnz2AG66djCrT/3KoV5uFDMrh1Tb5Rvjwqa45p4BUTtOA9gfeHufd13QKQB3qWpBVY8DeBJuYERERKa5DO4AACAASURBVJtftUrTvRX3uRLAlSLybyLyTRG5qW+n65HZTBoFW3H/Sc4hIqLwYkBE5LGcwYQWhiH4vR95Ma7dO9HX5/X1ZUi1KcWPL4DQhy4MiDpyL4BDIpIRkQSAmwHcVXGfv4VbPQQR2Q73hcCxXh5KwR4zIqIIaarSVERuFZGjInJ0fn6+z0dszcsOTkEEbDMjolBjQETkCXvbU6+ICOKm9HZIdcUMIisiM4iCLWZhPWvYqKoF4F0AvgDgMQCfUdVHROR9IvJD3t2+AOCCiDwK4CsA/ouqXujFebbW/81ERJHQtUrTbrQi98t4Ko6rd48zICKiUOOQaiLPVg4CEqbR0wois2IGUdircvx/A8Eh1Vvwn0XbVPVuAHdXXPdbgcsK4Je9P306U7+eiYiIGihWmsINhm4G8OMV9/lbuJVDH+9XpWk/zGTSuPPIc8hbTk9/MUdE1C5+ZSLyFAa05j4M4jGjpzOIYhtmEIV7dbx/LFYQRR+HVBMRhUvYKk37aTaTRrbg4KHTlwZ9FCKiqlhBROQpBQFb7xVlwjR622JmCgplM4gciLjzl8JIRGAaAkcVtvrVTgM+FBER0SYRxkrTfrjxYBqAO4foZZdNDfg0REQb8SUPkWerziACgOmxJLaPJnv2+JUVRAVHQx/EmSJeBZEbbJmsIIo0dpgREdGgbRtN4gU7RnHkeOSLoYhok2IFEZHHtrduK9En3zaLVLy/M4jC/nE2vVDL6zwMfaBF1QnHVBMRUYjMZNL4+wfOwHZ0S/5SkojCLdyv0Ij6yF/DvhVnEKVHEhhO9C4vdreYlVrMLDsCFUTFgMg9d1jb4YiIiCg6ZjNpLOcsPHZ2adBHISLagAERkcfawjOIei1mlreY2Y4DM+RBnB8Q8d/F5qBcY0ZERCEwk3HnEN3DdfdEFEIMiIg8YV+9HmVuBVFgi1kEZhD5c5Mcf80912FFEj9tREQUJrsnhnAgPcw5REQUSgyIiDwFb0ZOL9e9b1XVZhCFPYgzvFCLFUSbA+uHiIgoLGYyaRw5fpHVrUQUOnwlTOQpbatiENBtcbN8BlHBDv+QareCyClVloW8JY6IiIiiYSaTxsJaAU/PrQz6KEREZcL9Co2oj1gp0jumsXEGUdiHgRsisJ1A6yF7lYiIiKgLZjmHiIhCigERkYcziHqn2gyisH+c3cHaTvHcYT8v1ccqfiIiCosD6WHsHE/iCAMiIgoZBkREHn8GUYwziLouZhpQLYVwdgSGVJuGwFYUh1QzIIomYeUXERGFjIhgJrONc4iIKHT4SpjI488gCntwEUV+uOLPIXIriML95ceUigoiBg3Rxp+/iYgoRGYyaZxbyuLkxfVBH4WIqKipV2gicruIzInIw4Hr0iLyJRF5yvvvlHe9iMgHRORpEXlQRF7aq8MTdRNbiXrHD92iVkFk2QpHFYa4W80oevhZIyKiMCrNIeK6eyIKj2Z/hX8HgJsqrns3gC+r6iEAX/beBoDXAjjk/bkVwEc6PyZR71lcc98zftue38ZXsJ3QB3GmIXBUIzEviYiIiKLlBdOjmBqOcw4REYVKU6+EVfVrACq/er0BwF94l/8CwA8Hrv+Eur4JYFJEdnfjsES95FcQMQvovmoVRPGQbzHzB2vbDIg2BWWPGRERhYhhCG48mOYmMyIKlU5KJXaq6lnv8jkAO73LewGcDNzvlHcdUajZjoOYIRxq2wPVZxCF++NsGgLbD4j4byKy+KkjIqKwmsmk8dzFNZy9xDlERBQOXemlUXf8fku/nhWRW0XkqIgcnZ+f78YxiDoShdAiqvxqIb+Nz51BFO5WvrKAiP8uIo9LYoiIKGxmM9sAgG1mRBQanbxCO++3jnn/nfOuPw1gf+B++7zryqjqbap6WFUPT09Pd3AMou6wbOX8oR7xN5b5LWZRCOPMQItZjP8uIivc/8qIiGgru3rPOEaTMQZERBQanbzquQvAW7zLbwHwd4Hr3+xtM3s5gEuBVjSi0GKlSO/Eii1mfgWRE4ktZo7jDqk22KdEREREXWYagsMHpxgQEVFoNLvm/k4A3wBwlYicEpG3AfgDAN8nIk8BeI33NgDcDeAYgKcBfBTAz3X91EQ9YEUgtIiqWLHFzJtBZIc/jDMNw6sg4r+LzYAdZkREFEYzmTSemlvBhZXcoI9CRIRYM3dS1Vtq3PTqKvdVAO/s5FBEg2DZWgwyqLsqK4gsJ/ztfDFvzb3tIPRhFtXGofNERBRms5k0AODeEwu46dpdAz4NEW114X6FRtRHVgQGJ0dV5QyiKLTzGSKwbLeCKOxnJSIiomh68d5JJGMG28yIKBT4apjIE4XQIqr8yqyC32IWgbatYgWRIvRnpcaUa8yIiCiEEjEDLz0whSMnLgz6KEREDIiIfG4FEYOAXvA/rsUKokjMIJLiDCIj5Gel2thhRkREYTeTSePRM0tYyhYGfRQi2uIYEBF5LNvhDKIeMavMIAr7x9o0BLaj7mwqBkSRx/ohIiIKq9lMGo4C9z27MOijENEWx4CIyGM5WpyVQ93lD6S27FJAFPYKopgXEDnKNfdRxs8cERGF3Q0HphAzBPcc4xwiIhosvhom8thsMeuZUgWRv+beCf1AcMOvIIpAtRMRERFF11DCxHX7JnDkOOcQEdFghfsVGlEfFdhi1jMbZhBFIIzzK4g4vHxz4IxqIiIKs5nMNjx46hLW8/agj0JEWxgDIiJPFEKLqPIDlkKwxSzkYZxRHFKtMNliFl383BERUQTMXp6G5Sjuf45ziIhocBgQEXmiMBcnqvwZRNGrIHL472KTUI6pJiKiEHvZZVMwBLjnOOcQEdHgMCAi8rihBf+X6IXgDCJVjcRAcH+LmcOAKNL4mSMioigYT8Vx9Z5xHGFAREQDFO5XaER9xDX3vROcQeRXEYW9gsiU0pBqBkRERETUazMHt+Fbzy0gbzmDPgoRbVEMiIg8VgTanqIqFlhzb/kBUcjDONMU2Oquuee/i02AHWZERBRyM5k0cpaDh04vDvooRLRFMSAi8nBbVe/Eii1mEawgsvnvIso4o5qIiKLixoNTADiHiIgGhwERkcddc8//JXrBLLaYOcUKorDPIIp5W8wcZUBEREREvbdtNIlDO0Y5h4iIBibcr9CI+igKm7WiKu6FQQU7QhVEhgFVIG87DIg2AXaYERFRFMxk0jh6YqH48xIRUT8xICLycBhx75hmaUi15biDF8P+sfaLyQq2E/pqp7ARkZtE5AkReVpE3l3nfj8qIioih3t2Fu4xIyKiCJnJpLGSs/DomaVBH4WItiC+6iHysIKod4IziCw7OhVEAJC3nNCfNUxExATwIQCvBXA1gFtE5Ooq9xsD8AsA7unHuZS/iKX/v707j4/rru/9//rMomW0S15j2ZLsbDh7bJSkbAECSbh9JFC4fSS9LJeGhrbQ25bb2wuXX/lRWtoCbWj7awqkrJdSthRKCgmhCSEQmsRLFsdOYsex5X2VrX2Z7fv7Y86MjqSRpZE0Gp3R+/l46OGZM2dmvuec0Ryfjz6fz1dEJAA6O5oBeHJ/d4lHIiJLkQJEIp5EyqkHUZHkAkSpdC5lOigZRPFkmpA6HReiE9jrnNvnnIsD3wJuzbPenwGfAkaKORgdOhERCZLVDdWsa46pD5GIlISuhkU8qbQyRYol7M8g8gJE0UUejFMG0aytAQ757h/2luWY2dXAWufcjxZyYCIiIkFwTUczW7vOkFYfIhFZYIv7Ck1kAakHUfGYGeFQZtr4VEB6EGWDQqPJNKFFPtYgMbMQcBfwP2ew7p1mts3Mtp06dWpO7+vUplpERAKis6OZs0MJ9p4aKPVQRGSJUYBIxJNMuUWf1RJkkZCR8E1zv9izckK+rKfFPtZF5giw1ne/1VuWVQdcCvzMzLqAa4H78jWqds7d45zb7JzbvHz58lkNRkdORESC5pqOFgCeVJmZiCwwXQ2LeFLKICqqSMhIpcaaVC/2fe0PCi32sS4yW4ELzKzDzCqA24D7sg8653qdc8ucc+3OuXbgCeAW59y2Yg5KTapFRCQo1jZXs6q+Sn2IRGTBKUAk4kmqB1FRhUM2rgdRJLy493XYFCCaDedcEvgg8CDwAvAd59wuM/uEmd2y0ONRk2oREQkaM6Ozo5kt+7tx+guHiCygyGyfaGYXAd/2LVoPfAxoBH4LyDaM+D/OuftnPUKRBZBOO9JOgYBiioZDJNNpXw+ixR2f9n8WFDgsjPedf/+EZR+bYt3rF2JMIiIiQdLZ0cx9zx7l4Jkh2lpqSj0cEVkiZn2F5pzb7Zy70jl3JbAJGAK+7z382exjCg5JEARlZq0gyzapzpaYRRd50MWf4aQm1cGnv7+KiEiQXNPRDKgPkYgsrPm6Gn4j8LJz7sA8vZ7Igkqlg9EXJ8giISOZcoHZ1yFTBlE5MLWpFhGRADp/RS3NNRU8uU8BIhFZOPMVILoN+Kbv/gfNbIeZfdnMmubpPUSKJumVPSkQUDyRcChQPYjUpLq8qIeDiMjiYWY3mdluM9trZh8+x3pvNzOXb6bLcmdmvLK9iS1d3aUeiogsIXMOEHmz1NwCfNdb9DlgA3AlcAz4mymed6eZbTOzbadOncq3isiCyZY9KUBUPJFck+pg9CDyl5WF1ek4uHToREQWFTMLA3cDNwMbgdvNbGOe9eqA3weeXNgRLh6dHS0cOjPM0Z7hUg9FRJaI+bhCuxl4yjl3AsA5d8I5l3LOpYF/AjrzPck5d49zbrNzbvPy5cvnYRgis5fNagmrB1HRZHoQpQMTjBuXQbTIs51EREQCpBPY65zb55yLA98Cbs2z3p8BnwJGFnJwi0m2D9HWLpWZicjCmI+r4dvxlZeZ2WrfY28Dds7De4gUVbYvzmIPWgRZJBwKVA+isDKIyooqzEREFo01wCHf/cPeshwzuxpY65z70UIObLF5xep66iojalQtIgtm1tPcA5hZDfAm4P2+xZ82syvJTBrTNeExkUVprOxJgYBiGSsxy84Yt7j3dVg9iMqCjpyISLCYWQi4C/jvM1j3TuBOgHXr1hV3YCUQDhmb25vYogCRiCyQOQWInHODQMuEZe+a04hESiA39foiD1oEWdgLEI1lEC3ucj5/UEiZZSIiIvPmCLDWd7/VW5ZVB1wK/MwyGbyrgPvM7Bbn3Db/Cznn7gHuAdi8eXNZ5op2drTwyO4XOT0wyrLaylIPR0TK3OK+QhNZIMmABC2CLJLtQRSQcj5/WZkyiERERObNVuACM+vwJru5Dbgv+6Bzrtc5t8w51+6caweeACYFh5aKTq8P0Tb1IRKRBaCrYRHUg2ghRMJGIuVIpoJRzhcJ+wNE+qoMKlP/KBGRRcU5lwQ+CDwIvAB8xzm3y8w+YWa3lHZ0i89laxqoiobUh0hEFsScSsxEykXCC1ooQFQ8kVCI4UQqOBlEvqCQJrcLPjWpFhFZPJxz9wP3T1j2sSnWvX4hxrRYVURCXL1OfYhEZGHoskcEXwaRehAVzeQeRIt7X48vMdNXZVAt7k+ZiIjI9Do7mnn+WB99I4lSD0VEypyuekRQD6KFEA0byZSvB9EiT8tRk2oRERFZDDo7mnFOfYhEpPgW9xWayALJZrVEFQgomnDISKUdqXQwyvn8AaLQIh+rTM+hGjMREQmmq9Y2EQ2b+hCJSNEpQCQCgWmcHGSRUIhk2vmytRb3vlYGUXlQj2oREQm66oowl7c2qg+RiBSdAkQi4Ct70tVksUSyJWapoDSp9mUQKcoQeGpSLSIiQdbZ0cxzh3sZiidLPRQRKWMKEImAr3GyfiWKJdukOigZRBFlEJUFxfZERKQcXNPRTDLtePpgT6mHIiJlTFfDImia+4UQ8fUgCocMW+RX7v4AVliZZSIiIlJCm9qaCBnqQyQiRaUAkQia5n4hRMIhEqlMBlEQAnHjAkSLPJgl01OFmYiIBFldVZRLzmtgy/7uUg9FRMqYAkQi+HoQBSBwEVSZDKI0qVTwAkRBGK/kZ+jYiYhIeejsaObpgz2MJlOlHoqIlCkFiETwZRCpB1HR+HsQLfb+QzA+a0jT3AefU5dqEREJuM6OZkaTaZ473FvqoYhImdLVsAhjPYiCELgIqmg4RDLlSKbTRMKL/6vH33dIGUTBpepAEREpF69sbwbUh0hEimfxX6WJLAD1ICq+cK5JdTAyiPxBoSCMV0RERMpbc00FF66sZYsCRCJSJAoQFcFPdh3nln94LBd0kMUvKFOvB1kkZCTTaZIB6UEUMgWIyom+jUVEpBx0djSzresMSS/7XURkPilAVATPHellx+FeBuPJUg9FZih7ko2qB1HRREIh0i5TzheETC1lEImIiMhi09nRwmA8xfPH+ko9FBEpQ7oaLoLB0ZT3rwJEQZHLIApA4CKoskGh0WQ6EM3AwwoQiYiIyCLT6fUhUpmZiBTD4r9KC6AhL3MoGyiSxS+lae6LLhtkGU2mAxFwMTOyw9TnIvg0iZmIiJSDVQ1VtLXE1KhaRIpCAaIiGIxnAkNDKjELjKSmuS+6bJBlJJEKTMAl+3kI63MRWKZpzEREpMx0tjeztesMafU7FZF5pqueIhgaVQZR0CRTyiAqNn+AKAgZRADZuFBYQYYyoP9Ei4hIebhmfQs9QwleOjlQ6qGISJlRgKgIBnMlZsogmi8vHu/jkd0ni/b6qXQaMwgFJHARROFw5usm04MoGPs5l0Gk3lSBpSMnIiLl5pqObB+i7hKPRETKjQJERTDklZjNZhaznqG4StPyuPuRl3n//93Oqf7RKdfpGYozHJ9d1lYyHYyp14MsGrAeRECuB5EyiERERGSxaG2qZnVDlfoQici8U4CoCLKZQ0OzCFa85ytb+eSPXpjvIQVez1CceCrN1x/vmnKdd37pSf7qgdntu0yASL8OxRT29yAKB2NfZ8cZlICWTE1NqkVEpFyYGZ0dzWzZfwanE5yIzKM5X6WZWZeZPWdmz5jZNm9Zs5n9h5m95P3bNPehBkcug2gWJWZHzg5z8MzQfA8p8PqGEwD83ycOTJkldHgO+y6ZUgZRsY2f5j4Y+zobGArKeGUyJX+JiEg56uxo5mT/KAe6dd0gIvNnvv6M/3rn3JXOuc3e/Q8DDzvnLgAe9u4vGYNzaFLdP5LIBUNkTO9wgrXN1fQMJbh3+6FJjzvn6BtO0DvLfZdKp9VnpsiyM4EFqUl1trRMvamCT39fFRGRcjLWh0hlZiIyf4pV53Er8DXv9teAtxbpfRYd59ysp7mPJ9OMJtP0KEA0Sd9IktdesJwr1zbyxcf2k5owredgPEXaMet9pxKz4sv1IEoog0gWjqlNtYiIlKENy2tprqngCTWqFpF5NB9XxA74iZltN7M7vWUrnXPHvNvHgZUTn2Rmd5rZNjPbdurUqXkYxuIwmkznghcDBZaYZdefbRZMuXLO0TucoKE6yp2vXc+B7iH+4/kT49bJZl3NNvtKJWbFlw22xFPpXDbRYpcdc1AynkRERGRpMDM625uVQSQi82o+rtJe7Zy7GrgZ+ICZvdb/oMt0TpuU3e+cu8c5t9k5t3n58uXzMIzFwd+YutAm1QMjYwGidFoFEVlD8RSptKOhOsqNl6xibXM133jywLh1+kYygaGeocSsmvUl005BgCKL+Er4ghKMiyhAVDbUw1NERMpNZ0czh88Oc6RnuNRDEZEyMecAkXPuiPfvSeD7QCdwwsxWA3j/npzr+wSFvzF1oU2qs0EO56B/Fg2uy1U2o6q+Oko4ZFzR2siRs+NPhH3Dmf2VTLtZzR6XSqfHBTBk/vlL+IKyr3MZROp0HFg6dCIiUq46vT5EW5VFJCLzZE4BIjOrMbO67G3gzcBO4D7gPd5q7wF+MJf3CRJ/cGKwwB5E/pK03iGVmWVlA2cN1VEAGmPRSb2G/KVls+lDlEirxKzY/Ps3KPs6HDLM1KS6UGZ2k5ntNrO9ZjZpkgIz+5CZPW9mO8zsYTNrK/aYnNpUi4hImXnF6nrqqiI8qQCRiMyTuWYQrQQeM7NngS3Aj5xzPwb+CniTmb0E3ODdLxvnCt5kg0Jmhc9i1j8yFiDqGY7PbnBlKLu/66u8AFF1BT1D8XFleNkgEkDPUOH7LpVSk+pi85dpBakHUVCCWYuFmYWBu8mUHW8EbjezjRNWexrY7Jy7HLgX+HTRxlOsFxYRESmxcMh4ZXszW9SoWkTmyZyu0pxz+5xzV3g/lzjnPukt73bOvdE5d4Fz7gbnXNmEtV841sdVf/YTnj/al/fxIS8o1FJTUfAsZgOjY0EONaoe0+cFzvwZRGkHA779688gms2+Uw+i4ouEfSVmAdnX4ZARUo1SoTqBvd75IQ58i8zMljnOuUecc0Pe3SeA1gUeo4iISFno7Gjm5VODnB4YLfVQRKQMBOPP+IvIge5B0g52n8gfIMpmEC2vq5pbBpFKzHLGehBFgLFAkT+Tq3d4buV5qXSaaED64gSVPygUDsi+VgbRrKwBDvnuH/aWTeUO4IGijgg1qRYRkfKkPkQiMp8UICpQNnAzsUly1lAuQFRZcA8if4BIGURjstlBYxlEFcD4IJq/xEwZRItTOIA9iCIh0+eiiMzsncBm4DNTPH6nmW0zs22nTp2a5XvMYYAiIiKL3KXnNVAdDasPkYjMCwWICpRtgHykZyTv49msoeW1lblys5nqH0mSvRZVgGhMdl/UVY2VmAGc9fUa6htO5AJIs2lSnVQPoqKLjisxC8a+DpkCRLNwBFjru9/qLRvHzG4APgrc4pzLmxfvnLvHObfZObd5+fLlRRmsiIhIkFVEQlzd1sgWBYhEZB4E4yptEcllEPWcO4NoRX0l8VSaeDI949ceGM0EOaqiIQWIfPpGEtRVRnIX6o15AkF9IwlW1lcSDdus9l1KGURFNy6DKCAlZpGwBaah9iKyFbjAzDrMrAK4jczMljlmdhXwBTLBoZMLMSiVmImISLnqbG/hheN9un4QkTnTlU+Ber3ZxY5OESDKZhAtq60EKKhRdf9IkrqqaG6WLsnoHU5Q7wWFABpi2R5E/gyiJA3VURqqK2bVvymRTgcmaBFU43oQBSQYl8kgKvUogsU5lwQ+CDwIvAB8xzm3y8w+YWa3eKt9BqgFvmtmz5jZfVO83DwIxmdNRERktjo7mnEOtnUpi0hE5iZS6gEEzdnBsR5EzjlsQoOLoXiSWEWYusrMrh2Mp2iMzey1B0aS1FZGSEWd/gLg0zecHB8gymYQTehBtKq+iobqyLgZzWYqlXaB6YsTVP4AXFD2dSRkgSmHW0ycc/cD909Y9jHf7RsWfEwohUhERMrTVesaiYaNLfvP8MZXrCz1cEQkwHTlU6AeL4NoOJHKm6kyGE8Rq4gQqwxn7o8WkEE0mqSuKkJDLKpZzHz6hhPUV43FMisjYWIV4UklZvXVURpjFbljVIhkyqmUqMj8gZagZBCFQ6HAjFXyU5NqEREpd1XRMFe0NqpRtYjMma6IC9QzlMhlP+TrQzQ0mqSmMkxNNoOokADRiBcgqo4qg8in19eAOquxenwQrW84Sb2372YTXEulnaa5L7IgzmIWDgUnmCUiIiJLV2dHMzuP9BZ07SEiMpECRAXqHU5w/opaIH+AKJtBVFORCRANxWc+k9nAaMLrQaQAkV82O8ivMVaR6weVTjv6sxlEs9x3iXRagYAi8wfgIgFp7FNTGSFWES71MGQeqEm1iIiUs86OZpJpx9MHe0o9FBEJMPUgKlDPUILrNrTw4vF+jpzNEyAaTVJTEc5dVA4UmEFUWxmhMhJSiZlP3gwiXxneYDxJ2kF9VZT+6iS9s8wgCkpWS1AFMYPoj2+8uKDfYVl8gvFJExERmZvN7c2EDLbs7+bVFywr9XBEJKAUICrASCLFcCJFR0sN1dFw3pnMBuMpGqqj1FZmM4hmdnHpnGPAKzGLVYQZTqQYTaaojCzt7IVEKs1QPEV91eQA0Z4TAwD0jWT2cX11hMF4lP7RJMlUuqAsFfUgKr4g9iBa1VBV6iGIiIiITKu2MsKlaxrUh0hE5kRXxAXIzo7VWFPBeY1VU/cgqgj7mlTPrMRsJJEmmXbUen10AJWZMbbPG6rHxzL909mPrRPN7bts0Gimkum0ehAVWRBnMZPgmzjTpIiISLnqbG/m6UM9jCZn3uJCRMRPAaICZGfNaopFWdMUy5tBNDShB9FMG8X1j2Zeu64yQkOsAmBW07WXm2ygpyE2OYOodziOcy63n+qrojTGZhdcS6VdYLJagirsu1BXtpaIiIjI/OrsaCaeTLPjcG+phyIiAVU2V2lf/MU+3ve1rUV9j2zGSmN1BWumyCAajGdmMauOhjHLlJzNRL8XCKmrGsuCUR+isUDPpBKz6iiJlGMonvKVmEVprM4E13qGCpvqPqkeREUXChnZXax9LQtNTapFRKTcvbK9GYAtKjMTkVkqiwDR4bNDfPrB3Tyy+xSpdPGuArJBh8ZYlPMaqjk9EGckMT4ANDSaySAKhYxYNMzQDDOIBrwgR21lhEaVmOX4y8f8splCZ4fi4zKI6me571IpF5iZtYIsu48jKueTBaJPmoiILBVNNRVctLJOfYhEZNbK4or4Uz/eTTyZJpV2dA+MFu19enzBijVN1QDjysziyTTxVJoabwazWGWEwRk2qR7LIIrMawZRPJlm+4HgniRyGUQTAkQNuUyhhG+dyKxLzBLptLJaFkB2H6ucTxaaQylEIiJS/jo7mtnedYZkKl3qoYhIAAU+QLT9wFn+/dmjXLqmHoATfcULEGWnT2+MRTmvMRsgGsk9PuyVk8W8GcxqKyMzblI94PUgqq2afZAjn/uePcrbP/c4O48Esxa5b+TcGUS9w4ncOrWVs2/wrR5ECyO7jyPqQSQiIrJkmdlNZrbbzPaa2YfzPP4hM3vezHaY2cNm1laKcQZRZ0czg/EUu472lXooIhJAgb5Kc87x5z96nhV1lfyfO4eTFQAAIABJREFUt7wCgON9I9M8a/Z6huOEQ0ZtZYQ1XoDoSM9Q7vFstlCtN4NZrCI842nuc310qqLUef12euYhQPTSyX4A7n/u2JxfqxSm7EEUG8uy6htOUlsZIRIOzTr7Sj2IFoYyiGShaRIzEZHFxczCwN3AzcBG4HYz2zhhtaeBzc65y4F7gU8v7CiDq7NDfYhEZPYCHSC679mjPH2whz+68SLWL6sF4EQRA0RnhxI0VkcxM1Y1VBEyOOLLIMoGg2LeDGY1FQVkEPl6EIVDRn1VpKBZzHqHE/zeN5/mVP/4DKoDpzMBrB/vPI4LYJfWvuEkFeEQVdHxH9VcM+rhOH0jCeqrMvs8Gg5RUxEuKECUSjucQz2IFkCuB5ECRLLAAvj1JyJSrjqBvc65fc65OPAt4Fb/Cs65R5xz2b/CPgG0LvAYA2tlfRXtLTH1IRKRWQn0FXEq7Xj1+ct4x9WtLKutIGRwsogBot6hRG669Wg4xMr6Ko6cHetBlA0G1XgZRDWV4Rn3IBrwmlnXeoGOhli0oJm4tu4/w78/e5RH95wat7yre5BwyNh3epCXTg7M+PVm6tE9pzjeW8R9PpygvjqCTUgDGJ9BlBjXo6gxVlFQiVkynanRVlZL8eUyiNSkWhaIMohERBadNcAh3/3D3rKp3AE8UNQRlZnOjma2dp0hXcTJe0SkPAU6QPRrV7fy9Ts6CYWMSDjEstrKopeYNcUqcvfPa6we16R6cEIGUawywuAMZzHrH0lQFQ0R9TIsGqsrCiox6+oeBMZKyiBTgtfVPchbLluNGTzw3PEZv95MjCRS3PHVrXz+0Zfn9XX9+kYSkxpUA1RFw1RGQrkeRP4StPrqKL3DMw+uZWe+U1ZL8WWDcFH1IBIREZFpmNk7gc3AZ6Z4/E4z22Zm206dOpVvlSXpmo4WeocT7PFdF4iIzETgr9L8mSUr66uK2qS6xysxyzqvsZojvgDRUDaDKFdiFmYoPtMm1clc7yHINGUuJAvmQHcmC/dlX5bQyf5RRhJpOtub2NzWxAM757cP0d6TAyTTjj0ninfy6RtOTOo/lNUUq6BnKE7fcJL66khueWOB+y6ZDRCpxKzosgFQZWvJQtPfUEVEFo0jwFrf/VZv2ThmdgPwUeAW51ze/+A75+5xzm12zm1evnx5UQYbROpDJCKzVVZXxJkAUREziHwlZgBrGqs51jucS9/MZRDlmlRHcqVj0+kbSVJXORbkaIhFc7OmzUQ2g2ivL0DUdTqzrK2lhhsvWcWLx/tzy+bD7uOZwFAxStey+oYTk2Ywy2qMRTMlZhMyiBqqowX1IEqmlEG0UHKzmKnETBaIoc+aiMgisxW4wMw6zKwCuA24z7+CmV0FfIFMcOhkCcYYaK1N1ZzXUKU+RCJSsFkHiMxsrZk94k1BucvMft9b/nEzO2Jmz3g/b5m/4Z7byvpKTvYXcZr74USuOTLAmsYqEinHqYHMe2azhbIZRLWVEYbiqRk1hx4YSVJX5QsQFZgFkw0QHTwzxEgiM45sVlF7Sw03XboKgB/vmr8ys2zm0Kn+0YKnlZ+p3uH8JWbgBYKG8/UgKjSDSD2IFopmMZNSCWKTfhGRcuScSwIfBB4EXgC+45zbZWafMLNbvNU+A9QC3/WuJ+6b4uUkDzOjs6OZLfvP6PwnIgWZSwZREvifzrmNwLXAB3xTVH7WOXel93P/nEc5QyvrqzgzGGc0ObOyrkIkUmkGRpO55sgArU0xAA6dyQRisv2GchlElWFSacdoMj3t6/ePJHINqiFTJtUznJjRl3o8mebI2WHWL6sh7WC/lyXU1T1IJGSc11hFa1OMy1sbeGDn/AWIXjw+Vlq2dwZZRM45dh7pLehE1TucoMFXPubXGItydjBO/2gyN4sZeA2+pwkQOed44Vgfzjn1IFpA2cwh7WtZMPqoiYgsOs65+51zFzrnNjjnPukt+5hz7j7v9g3OuZW+64lbzv2KMlFnRwun+kfp6h6afmUREc+sA0TOuWPOuae82/1k/gJwrhkIim5VfRUAJ4vQhyibkeIPELW1ZAJE2S/e7Cxmsag3i5mXSTSTRtUDo0nqKseXSaXSjsEZ9DA6fHaItIM3vmIFMBasOdA9xNrmWK63zptesZJnD/UUVLp2LntO9LOprcl7z+n7EP3LloP86v/32IzroZ1z9I0kp+xB1FhdwZGeYZxjXAZRQ3WUeDKdy6SaKJV2/MkPdnLz3/2Cr/5n11iJmXoQFV04pB5EIiIiIsU21oeou8QjEZEgmZcrYjNrB64CnvQWfdDMdpjZl82saT7eYyZW1FcCcLJ/7n2ITvaP8OpP/ZSfe9PGZ3va+PvhtDbFCIeMA15511A8SWUklAs01Hg9hWbSqLp/JDk+gyg3jfv0s3FlS8lef9EKQjbWE2j/6cFcEAvgwlV1QKYMze+Lv9jH7fc8Me37+PUOJTjWO8IbLl5BZSQ0bQZR30iCu36yB4DtB8/O6D0G4ylSaXfOHkTZfesPImXLAHuGEgyOJnnD3/yM3/inJ3j4hRMMjiZ5/9e3889PHKQxFuWLv9ifyzhTVkvxZfdxRLOYyQJTgr2IiCwlG5bX0FJTwZP71IdIRGZuzldpZlYL/CvwB865PuBzwAbgSuAY8DdTPG/ep6Vc6WUQHe+dewbR1/6zi8Nnh/npi5m+eNlp0xt909xXREKsaaweyyCKJ3NBIcjMYpZdPp18PYgy7zt9tk+2/9BFq+pY2xzj5ZMDOOc40D1Ie0tNbr11zZlg0YEz4xtVP7b3NI/v6+bw2ZmnoGanzdy4up71y2unbVR990/30j0Yp64qwo5DvTN6j77hyUE5P3/D8IkZRJDZd9/ddoh9pwbZc2KAO762jc1//hAPv3iCT9x6CZ9+++Uc6Rnm35/NzO6mrJbii6hJtSwwfdJERGQpyvYhUqNqESlE/uYuM2RmUTLBoW84574H4Jw74Xv8n4Af5nuuc+4e4B6AzZs3z8sfd7MlZnOdyWwonuSfnzgIwI7DPcBYBlHjhGBFW0tsLINoNEXMCwoBxCpnVmKWTjsG4hNmMfOyYGZSDtZ1epC6ygjNNRVcsKKWvScHOD0QZzCeot2XQbTWCxBNzCDKZiA9ue8MrZtizES2/9CFq+q4YEUtT50jK+hg9xBf+WUXb7+6ldFkiqcP9szoPbLBsamaVPsbho+b5t4LHHUPjvKlX+5nU1sT37rzWu5/7hjff/oIt3eu48ZLVpFOOzqW1fC1x7sAiCpoUXTqQSSloh6dIiKy1HR2NPPAzuMcPjuU650qInIuc5nFzIAvAS845+7yLV/tW+1twM7ZD68wjbEoFeEQJ+ZYYvbdbYfpHU6wqa2JXUf7SKTSuQBRky+DCDIzhO0/PYhzjsF4klpfkKfWa1ad7U00lcF4Euegrip/Fsx0urqHaFsWw8zYsKKWfacHePlUJqOnbdlYBlFtZYSWmopcU22AZCqdu//4vpnXKO853k9dVYTzGqo4f0UtR3qGGZoiU+qvfvwC4ZDxxzddxBWtjRzpGeb0QP4sL+ccw17Z2HQZRP5+UBOnuQf4ztZDHDozzG+9Zj3RcIhbr1zDV9/byY2XZGZ0C4WM972mI3dswyp7Kjr1IJKFljlViYiILD3ZPkRbu5RFJCIzM5cr4lcB7wLeMGFK+0+b2XNmtgN4PfCH8zHQmTAzVtRXzqlJdSrt+OJj+7h6XSPvvq6N0WSal04M5GbF8pc1QSaDqH8kydmhBEPxCRlEFdkeROfOIOofyTyetwfRDAJE/lKy85fXkkg5fvFSpmzPX2IGmSwifwbR0Z4RkmlHNGw8UUCAaPfxfi5aWYeZcf6KWpyDfacGJ62350Q/9z93nPe/bj0r66u4vLUBgOcO5y8z+8ovu+j85EPsPTkwlkE0ZZPqyUEh/+0fPHuUtpYYb9q4csrtePvVrTTXZIJ+ymopvqh6EImIiIgsiItX1VNXFZnxBDEiInOZxewx55w55y73T2nvnHuXc+4yb/ktzrlj8zng6ayqr+J47+wziB7cdZxDZ4a587Xruby1EciUmfUMxQkZ48rAADq8DJ2u7kEGRyf2IMrcHpgmg2jAK0HL14OoZ5oSs0QqzeGzw7lA0AUrM42oH3r+JOGQsaaxetz66yYEiLL9i968cRWHzw6Pyy66+5G9/OPP9k56T+ccu0/055peX7CiFsg/1f1jL50G4Nc3rwXgkjUNmMGzh/OXmT384gn6R5N84BtPcaJ/dNy+mKhhqgwib7lz8L5Xd5wzW6UqGuZd17YBympZCNl9rH0tC081ZiIisrSEQ0Znu/oQicjMld2f8VfWV+UtMRuKJ/n21oPTZsnc8/N9XtbJKtpbYtRXRXj2cC89QwkaqqOEJlzYtnmBmQPdg5MyiGq8ErPpM4gyQSB/eVqsIkw0bNOWmB3tGSaZdrnZyjYsz4xn94l+1jRWUxEZf4jXNcc42jNCIpUGxgJEt3VmAjjZ/XO8d4S/fWgPX36sCzeheceJvlF6hxNc7AWI2lpqCIeMl/JMdf/4vm7aWmKc5wWqaisjnL+8lh15MohGkym2HzjLFWsb2XOyn7t+shsY31/Iz98w3J99VVcZIRwymmJR3rFpbd7n+v3mqzu449UdXLWucdp1ZW7Ug0gWmj5pIiKylHV2NLPv1CCn+uc+iY+IlL+yCxBNLDHrHhjl0z9+kev+8qf87399jju+ujXXn2eirtODPHOoh3dd20Y4ZJgZl7c28tyRHnqGE+MCEllrm6sxg67TQ5lZzCp8GUS5JtXnziDKlpj5exCZGSvrq8Zl9OSz/3QmwNPuZTLVVUVzzbrbl9VMWn9dc4xU2nGsZ8Tb5iGqo2FetWEZzTUVPOFNhfnV/+wikXKcHhjl8Nnhca+x+4TXoNrLVqqIhGhriU3KIEqnHVv2n+HajpZxyy9rbWDH4d5Jgacdh3sZSaT5nddt4APXn89ZL3uqbpoSs2xAKMvM+JUNLfyPN15AtS9gN5WG6ih/8qsbp3wfmT/Z0jLNYiYiIiJSfOpDJCKFKLsA0ar6KgZGk7myrd/+5+18/tGX+ZUNLdzzrk1URsN84BtP5Roh+z26J9O3x9+z5rLWBl481s+JvpG8pU6VkTDnNVRnMohGU8Qqw77HQoRs+lnM8pWYAVy1rumcs4PB2Axkbb7Zyi5YmSn58s9gljVxJrMD3YO0tcQIhYxrOpp5Yl83A6NJvvHkgVyG0PYD48ew+3gfABd5ASIgN3ua3wvH++gdTnDthuZxy69obeT0wCjHJpQCPv5yN2Zw7fpm/uCGC7h2fTMr6iqnLEeKVYSpCIfyznL29Tuu4b2v6sj7PCmdiHoQSYloFjMREVmKLl3TQHU0rD5EIjIjZXeVttI31f3ek/1s7TrLh2++mM+9cxNvvmQVd/36Fbx4vJ+P37dr0nN/tvsk7S2xXNkYwBWtDSTTjmcO9YybNcuvY1kN+7snZxCZGTWVEQZn2KR6YoBo07pGjvWOcLRnON/TgEyJWE1FmOW1lbllG5ZnAkRtLXkyiFrGB4i6fA2ur9vQwpGeYf7mJ7vpH0nyybddRm1lJE+AaIAVdZU01YxlVJ2/opau7iHiyXRu2eMvZ8rVrl0/OYMImFRm9sS+bi5eVU9jrIJIOMRX39vJ9373V6bcdjOjIRadtN9k8VIPIllomsRMRESWsmg4xKa2JvUhEpEZKbsA0Yr6TKDkRN8I391+mHDIeNtVrbnHr79oBR94/Qa+ve0QP3jmSG75SCLF4/u6ed2Fy8e9XrZRdTyZHjdrll9bS4x9pwYYSaRzM5dl1VREGJquSXV2FrMJDbA3tWUyb/wBmlTa8fcPv8QzhzJNng90D9HWUjNuKudzZRCtqq8iGjYOnhkilXYcOjNM27LMetlAzld+2UVnezOb2pq4al3j5ADRiT4uWlU3btn5K2pJpR0HusdmMnti3xnaW2KsbhjfKHvj6noiIWOHr1F1tv/QtevHso2qomFamyZvg19jdTRvBpEsTpGwV2KmAJEsMCUQiYjIUtXZ0cyLx/vonWbyGxGRsgsQZfvvHO0Z4ftPHeH1Fy1neV3luHX+8IYLuby1gb/+yW5S6cxlw5b9ZxhJpLn+ohXj1l3dUMWy2kymTL4eRJCZSj6bBVRTOb7nTawyzMAMmlSbMS77CODi1XVUR8PjAjTbus5w13/s4bZ7Hucnu45nMoCWjQ+ivO7C5bzmgmVcva5p0nuFQ0ZrU4xDZ4Y41jtMPJWmIzsD2opaWrysoN967XoArl7XxIvH+3JlcMd6h9l1tG/Sa1+wIhMweskrM0ulHVv2d0/KHoJM4OfClXXjMoieOdjDaDLNdXnWP5e3XrWG/3LZ6oKeI6UTCRlmTGr2LlIspjbVIiKyxHV2NOOc+hCJyPTKLkC0wgsQ3bv9ECf7R/POYhUJh/jd6zdw6MwwD+46DmT6D1VEQlyzfny/nGyjamDKEjN//5+JGUS1lRGGpulB1D+apLYiMumiORoOccXahnF9iB7YeZzKSIiLVtbx2/+8na7Tg5NKyVqbYnz9jmvGlYD5rfWmuu86ne1fVJPb1jdcvIKLV9XxxoszgbJNbU2kHTzrZSx976kjOAe/dvWaca+5YXktVdEQ//LkQVJpxwvH+ugbSeYNEAFcsbaBHYd7co2qn9h3BrOxRnoz9YHXn897fqW9oOdI6YRDpuwhERERkQV05dpGKsIhtihAJCLTKLsAUW1lhNrKCE/sO0NzTQVvuHhF3vXetHEVbS0x7vn5PpxzPLrnFNd0NE8K8ABctibTM2eqEjP/bGGTMogqwgzmaYjt1z+SnLKPzqa2JnYd7WMonsQ5x4O7jvPaC5fzzTuv5fUXrSDtYH2e2crOZV1zdSZA1J2dAW0swPUXv3YZ//aBV+WCVVeua8QsU+bmnOPe7Yfp7GieFJSqrgjzsV+9hMf2nuYfH9nLE/vy9x/KuqK1kb6RJPduPwxk+g+9wus/JOUrVhGmOjr9zHIi8yXbDz2ZGl9kdqRnmF1He/M8Q0REpLxURcNcsbZBfYhEZFpl2d13ZX0lA6eS3HrleVRE8sfAwiHjfa/u4E9+sIv7nj3K3pMD3PbKydlGkMl2galLzNY1xzDLzJKTrwfR8b4RnHN8+ZddHO0Z5qNvecW4bKGBkSS15wgQpdKOHYd7qYqGOdY7wh+9+SJiFRG+8K5NPLL7FNdftDzvc6fS1lxD73CCHYd7qIyEWFlXlXssGg7hv36vr4py4Yo6th84y1MHz7L/9CC/c/2GvK97e+dantzfzWcf2sO65hgdy2pY1VCVd91brjyPHzxzlP917w6O9ozw1MGzvPPatoK2Q4Lnva/qmDJoK1IMld45IJ5Kj1v+qr/6KQBdf/VfFnxMIiIiC62zo5nPP7qPwdEkNZVleQkoIvOg7DKIYGwms3dsaj3neu/YtJamWJT/873nAKYMtFy3fhm3d67lV87Pnw1TFQ2z2nvPmoqJPYgi9I8k+X/+bSd/9sPn+dJj+7nnF/vGrdM3kpjUoDrrqrWZXj9PHTzLAzuPEQkZN7xiJZAplXvTxpVEw4UdxuxU94+9dDo3xf25XN3WxFMHz/LtrYeIVYSn7PljZvzF2y6jfVkNXd1D4xpOTxSriPDV33wlt155Hp99aA+jyfSU2UZSPpbXVbK5vbAyQpG5qIxkvpP9MyyKiIgsNZ0dLaTSblzrChGRicoyQLS5vZnXXLCMS85rOOd61RVh3nltG4PxFGsaq3PTw+db7y9/7XJW1OXPhoGxPj4TI/K1lWEOnhniG08e5Ldft4G3XLaKzzy4O9ck7qu/3M/j+7q5cGXdpNcEaKqpYMPyGrZ3neXBnce5bkMLDVP0QpqpdV6A6GjvSG6K+3PZ1NZE/0iSf33qCDdfuvqcf3WoqYzwj//tapbVVnLjJavO+bqVkTCf/fUr+d3rN9CxrGZS/ycRkbnKZhCNJs9d6jvRm+56lC9OCObLwnlk90n+5N92lnoYIiJlY1NbE+GQsUVlZiJyDmWZX/ihN10443XffV079/x8H2+4eMW4qeIL1b4sxuP7uif1IGqKVWAGf3rLJbz7unb6RhLsOvoYv/cvT3PjJSv52uMHePPGlXz8lkumfO2r1zXxg2eOEk+lufO1+cu7CrG2eWza+fYZ9C/a1JbJYkqlHf9187mzsgAuXlXP1o++cUb7MxQy/vimi/njmy6edl0RkUJly4xHE4VlEL10coA//9ELvO8164sxLJnGe7+yFYA/e+ulJR6JiEh5qK2McOl59epDJCLnVJYZRIVYXlfJj/7Ha/hfN100p9fJZuJM7EH029dv4Me//1refV07kOnpc/dvXM2ZoThfe/wA776ujc+9cxNV52jcu6mtiXgqjRm8+ZKVcxonQF1VlGZvhjP/DGxTaW+J0VxTwdrmajpnWB40l2CbiASbmd1kZrvNbK+ZfTjP45Vm9m3v8SfNrL1YY8kGiLYdyJ9SP9O/pO49OcDnfvbyvI1rrk72jfDvzx4t9TCmtPdkP//w05dKPQyeP9rH6YHRUg9j1kYSKb731OHcjJ+y8K7/zCP86b/vKug5H/neDt7/9W1FGpHI7HV2NPPMoR5GEoVl1YrI0rHkA0QA56+opb5qbmVbb7t6DR+5+WJWT2jKXF8V5aJV48vHLl3TwOffeTWffsfl/OktlxCepgdQNoPnle3NLKutnNM4s7J9iGZSYmZm/PlbL+Uv33b5tP2KRGRpM7MwcDdwM7ARuN3MNk5Y7Q7grHPufOCzwKeKNZ5UOnNh/c0tB/M+/utfeJxT/WMBhB/uOMrfPTQ5sHHDXY/yqR+/yN6TA8UZ6BSG4kkGR5OTlr/17l/ye998Ou9j8613KMHZwXju/sBokmTq3BlZt93zBH/9kz189PvP0fGRHwHgnGNwBs+dT2/5+1/w5s/+PHffOTejYMvekwP8fM+pgt8vnXZ8a8vBeel5dXYwzke+9xwf+s6z/OKl03N+vUINFOmzdbB7iENnhma8/pGe4Rmtn0o7hqeZNRYgkUpzsn9kxu/f1T3EV37ZBWQ+Pwe8GWCdc4wmU7x0oh+An+85xR9991kAvrnlEA/uOgHAnhP9eT8P39l2iG2aclwWWGdHC/Fkmh2HNYuniORXliVmpbCiror3v27m5V9vuHjmmUAbltfyuguXc3tn/lnWZmNdc4xnD/XMKIMI4C1TNKYWEZmgE9jrnNsHYGbfAm4Fnvetcyvwce/2vcA/mJm5IqRJrKofC9p/76nDxCrChCZkOL7ykw/x/teuJxI27n5kfJbQ3z60Z1wg/Ya7HuXP33opvcMJnHP89U/2AJlJEe7dfphf39zKFWsb+buHXuJk/yjvua6N+uooqxqqiCfTDCdS9I8kqQiHeOFYHz1DCbbkuUhcVV/F8b7JF7E3vGIl121o4Whv5rFL/t8HWVZbyavOb+HClXV85Zf7OT0wFswxg5svXcX9zx2fch+ta47RMxQnFDJ+6zXrGU2m+fuHM0GyS86rZ9fRvnM+d1NbE99/+ggAdd6MnP0jmeDCN57MBObaP/yjSc+tCId4+6ZWHth5jJ6hRN7Xzz7vwpW1VFdEuHxNA/0jCRq98u29Jweor46yYVkNEW/Chngyzc/2nGTnkcy4zwzG+Yv7X6A6GubvHh4L/q1prCaeSrO6oYodh3tZXlfJTZesYv/pQR7bOz4gs6y2ctpMpLdcNrafP+xNfpFlBhtX19OxrIb/fLmbM17Ara0lRjQcYnVDFS+dGGAonqRvZHJg5t1f3jLu/k2XrOLSNfV86bH9XLyqnsf3dQPwzmvX8c9PHKShOkrHshqeOdRzzjFfu76ZJ/adYV1zjEjI2Hd6cNptg8wf1vaeHMh9PjJZxjEuPa+eltpKvv/0YQ6dGR73Grd3rsM5x7e2Hhq3/IrWBp6dcLEaCRnJ9Pivg4pwaNJshLnXWNvIs9Ns61zl+wznc+/2w1M+Z1ltJnvb/zvq99j/fj2tTTP7f5nIbLyyPfNH5+9uO0TvcP7vXRFZvK7b0DLl5FbzxRZD2vLmzZvdtm1KxV1In3/0Zf7p5/vY8tEbps1gEpHSMrPtzrnNpR7HTJjZO4CbnHPv8+6/C7jGOfdB3zo7vXUOe/df9tY5PeG17gTuBFi3bt2mAwcOzGpMl3/8wbwX3VJ6DdXRWV+k1FVFcoEov3DIcpljC6m1qZreoQT9U2TeRMNGfVWU7sH8wQGZvcZYdMog43xpqako+rH7/Ds3cdOl557gI58gnSOKSdcTM3PLPzymDCKRgHroQ6/j/BX5J9Y6l0LOE8ogWqLueHUHt71yrYJDIrJoOefuAe6BzH/8Z/s6Oz5+Iyf7RugZTpBKO1JpR9o5BkaTubKUtpYaBkYT1FVFSTtHOg1NNZnS4+F4ipAZQ/FULkOmdzhBc00F8WSmP1xdVZSBkSSV0RCptKMyEmIonsp9x4bMqIiEqIiEGIonSaYcFZEQiVSas4MJKqMhGqujJFIOM6iKhImn0oQsU+YbNiPlXOY+mds1lWEqwpn3cYBB7t9wyBj2ljvniIRCxCrDGJB24HCEvUyqcMhIpDKvnUw7wiEjErJxveSccyXtLTfV+6fSLrfdyXSasFkukyhTTgYp54iEjLQjtz+dcyS8Y7CYZMdsRm68AIlU5uPvcIwk0hzoHqTB+7xURkKknSOeTFMVzXxuOlpqciXhzrnca6Vd5vMBmffI7gvnMvvQv3+AXBZPyIy0y3xmsj0Rw5b5jGR/n4bjKYYTKSoiIaLhEGaQTGV+3yoiIdJpR+9wgpTLLFvTWH3OWVFFpDi+/pvXcOjszMs8RWTxaG2qnn6lOdKZeYmKhkM0xipKPQwRKT9HAH89bKu3LN86h80sAjQA3cUc1Ir6KlbUV02/4gzlK/jNNv/PapniuRNTg1c3zO1kP9VF9rkmP5ioIpIJG0SmeEqpJx6Y6v00455xAAAJCUlEQVT9f+QIh8YP3swwg5AXEgnb+Mey27yYZMcM48frH2tlJMzlrY0FvWY4z2vme0//MshkPWWFvf1YNWE/Z49BVTRM0zRjaarR/ztESq0hFqUh1lDqYYjIIrW4/nQmIiJBtxW4wMw6zKwCuA24b8I69wHv8W6/A/hpMfoPiYiIiIjIzCmDSERE5o1zLmlmHwQeBMLAl51zu8zsE8A259x9wJeAr5vZXuAMmSCSiIiIiIiUkAJEIiIyr5xz9wP3T1j2Md/tEeC/LvS4RERERERkaioxExERERERERFZ4hQgEhERERERERFZ4ooWIDKzm8xst5ntNbMPF+t9RERERERERERkbooSIDKzMHA3cDOwEbjdzDYW471ERERERERERGRuipVB1Ansdc7tc87FgW8BtxbpvUREREREREREZA6KFSBaAxzy3T/sLRMRERERERERkUWmZE2qzexOM9tmZttOnTpVqmGIiIiIiIiIiCx55pyb/xc1uw74uHPuRu/+RwCcc385xfqngAOzfLtlwOlZPjdolsq2LpXthKWzrUtlO6E429rmnFs+z68ZKDpPFETbW960veVrttu65M8RoPNEgbS95W0pbe9S2lZYgPNEsQJEEWAP8EbgCLAV+A3n3K4ivNc259zm+X7dxWipbOtS2U5YOtu6VLYTlta2BsVSOyba3vKm7S1fS2lbF5ultu+1veVtKW3vUtpWWJjtjRTjRZ1zSTP7IPAgEAa+XIzgkIiIiIiIiIiIzF1RAkQAzrn7gfuL9foiIiIiIiIiIjI/Stakeh7dU+oBLKClsq1LZTth6WzrUtlOWFrbGhRL7Zhoe8ubtrd8LaVtXWyW2r7X9pa3pbS9S2lbYQG2tyg9iEREREREREREJDjKIYNIRERERERERETmINABIjO7ycx2m9leM/twqcczX8xsrZk9YmbPm9kuM/t9b3mzmf2Hmb3k/dtU6rHOBzMLm9nTZvZD736HmT3pHddvm1lFqcc4H8ys0czuNbMXzewFM7uujI/pH3qf3Z1m9k0zqyqH42pmXzazk2a207cs7zG0jL/3tneHmV1dupEvXeVwnij0nHCuz56Zvcdb/yUze0+ptmkmZnpuMLNK7/5e7/F232t8xFu+28xuLM2WTK+Q80M5HN9CzhFBPL7zda6Y6nia2SYze857zt+bmS3sFpYXnSeC+T0COk+U8/HVeaKE5wnnXCB/yMyO9jKwHqgAngU2lnpc87Rtq4Grvdt1wB5gI/Bp4MPe8g8Dnyr1WOdpez8E/AvwQ+/+d4DbvNufB36n1GOcp+38GvA+73YF0FiOxxRYA+wHqn3H87+Xw3EFXgtcDez0Lct7DIG3AA8ABlwLPFnq8S+1n3I5TxR6Tpjqswc0A/u8f5u8202l3r5zbPeMzg3A7wKf927fBnzbu73RO+aVQIf3WQiXerum2NYZnx+CfnwLPUcE8fjOx7niXMcT2OKta95zby71Ngf1B50nAvk94ttunSfK8Pii88ScjydzOE+UfOfMYadeBzzou/8R4COlHleRtvUHwJuA3cBqb9lqYHepxzYP29YKPAy8Afih9yE+DUTyHeeg/gAN3hedTVhejsd0DXDI+7KKeMf1xnI5rkD7hC/zvMcQ+AJwe7719LNgx6oszxPTnROm+uwBtwNf8C0ft95i+ink3AA8CFzn3Y5469nE4+1fbzH9FHp+CPrxLfQcEdTjO9dzxVTH03vsRd/ycevpp+DjpPOEC973iDc2nSfK9PjqPFHa80SQS8yyH5ysw96ysuKlyF0FPAmsdM4d8x46Dqws0bDm098CfwykvfstQI9zLundL5fj2gGcAr7ipcJ+0cxqKMNj6pw7Avw1cBA4BvQC2ynP4wpTH8Ml8R21yJXdMZjhOWGq7Q7S/ijk3JDbLu/xXm/9oGxvoeeHQB/fWZwjgn58s+breK7xbk9cLrMTtM/RtHSeKMvvEZ0ndJ5YsPNEkANEZc/MaoF/Bf7AOdfnf8xlwoGuJAObJ2b2q8BJ59z2Uo9lAUTIpBF+zjl3FTBIJnUwpxyOKYBXL3srmZPZeUANcFNJB7VAyuUYyuJU7ueErCV2boAldH6ApX2OyCqn4ymLi84TZUvnCZ0nFkyQA0RHgLW++63esrJgZlEyX/DfcM59z1t8wsxWe4+vBk6Wanzz5FXALWbWBXyLTIro3wGNZhbx1imX43oYOOyce9K7fy+ZL/pyO6YANwD7nXOnnHMJ4HtkjnU5HleY+hiW9XdUQJTNMSjwnDDVdgdlfxR6bshtl/d4A9BNcLa30PND0I9voeeIoB/frPk6nke82xOXy+wE7XM0JZ0ndJ6gfI6vzhMlPE8EOUC0FbjA62ZeQaYh1X0lHtO88LqMfwl4wTl3l++h+4D3eLffQ6a+OLCccx9xzrU659rJHL+fOuf+G/AI8A5vtcBvJ4Bz7jhwyMwu8ha9EXieMjumnoPAtWYW8z7L2W0tu+PqmeoY3ge825t54Fqg15c2KgujLM4TszgnTPXZexB4s5k1eX+de7O3bFGZxbnBvx/e4a3vvOW3ebObdAAXkGnauKjM4vwQ6ONL4eeIQB9fn3k5nt5jfWZ2rbf/3k35nE9LQeeJAH6P6Dyh8wRldHx9Fsd5YqbNihbjD5mO3nvIdCT/aKnHM4/b9WoyKWU7gGe8n7eQqaV8GHgJeAhoLvVY53Gbr2dsBoL1ZH559wLfBSpLPb552sYrgW3ecf03Mt3my/KYAn8KvAjsBL5OZvaAwB9X4JtkaqETZP6ac8dUx5BMc7y7ve+n54DNpR7/Uvwph/NEoeeEc332gN/0fgf3Au8t9bbNYNunPTcAVd79vd7j633P/6i3H3aziGd6KuT8UA7Ht5BzRBCP73ydK6Y6nsBmb9+9DPwDExrX6qfg46XzRAC/R3zj1XmiDI+vzhOlO0+Y9wIiIiIiIiIiIrJEBbnETERERERERERE5oECRCIiIiIiIiIiS5wCRCIiIiIiIiIiS5wCRCIiIiIiIiIiS5wCRCIiIiIiIiIiS5wCRCIiIiIiIiIiS5wCRCIiIiIiIiIiS5wCRCIiIiIiIiIiS9z/D8AXV0UYEMl+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1440x360 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "agent.train(num_frames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test\n",
    "\n",
    "Run the trained agent (1 episode)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "score:  200.0\n"
     ]
    }
   ],
   "source": [
    "frames = agent.test()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Render"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<script language=\"javascript\">\n",
       "  /* Define the Animation class */\n",
       "  function Animation(frames, img_id, slider_id, interval, loop_select_id){\n",
       "    this.img_id = img_id;\n",
       "    this.slider_id = slider_id;\n",
       "    this.loop_select_id = loop_select_id;\n",
       "    this.interval = interval;\n",
       "    this.current_frame = 0;\n",
       "    this.direction = 0;\n",
       "    this.timer = null;\n",
       "    this.frames = new Array(frames.length);\n",
       "\n",
       "    for (var i=0; i<frames.length; i++)\n",
       "    {\n",
       "     this.frames[i] = new Image();\n",
       "     this.frames[i].src = frames[i];\n",
       "    }\n",
       "    document.getElementById(this.slider_id).max = this.frames.length - 1;\n",
       "    this.set_frame(this.current_frame);\n",
       "  }\n",
       "\n",
       "  Animation.prototype.get_loop_state = function(){\n",
       "    var button_group = document[this.loop_select_id].state;\n",
       "    for (var i = 0; i < button_group.length; i++) {\n",
       "        var button = button_group[i];\n",
       "        if (button.checked) {\n",
       "            return button.value;\n",
       "        }\n",
       "    }\n",
       "    return undefined;\n",
       "  }\n",
       "\n",
       "  Animation.prototype.set_frame = function(frame){\n",
       "    this.current_frame = frame;\n",
       "    document.getElementById(this.img_id).src = this.frames[this.current_frame].src;\n",
       "    document.getElementById(this.slider_id).value = this.current_frame;\n",
       "  }\n",
       "\n",
       "  Animation.prototype.next_frame = function()\n",
       "  {\n",
       "    this.set_frame(Math.min(this.frames.length - 1, this.current_frame + 1));\n",
       "  }\n",
       "\n",
       "  Animation.prototype.previous_frame = function()\n",
       "  {\n",
       "    this.set_frame(Math.max(0, this.current_frame - 1));\n",
       "  }\n",
       "\n",
       "  Animation.prototype.first_frame = function()\n",
       "  {\n",
       "    this.set_frame(0);\n",
       "  }\n",
       "\n",
       "  Animation.prototype.last_frame = function()\n",
       "  {\n",
       "    this.set_frame(this.frames.length - 1);\n",
       "  }\n",
       "\n",
       "  Animation.prototype.slower = function()\n",
       "  {\n",
       "    this.interval /= 0.7;\n",
       "    if(this.direction > 0){this.play_animation();}\n",
       "    else if(this.direction < 0){this.reverse_animation();}\n",
       "  }\n",
       "\n",
       "  Animation.prototype.faster = function()\n",
       "  {\n",
       "    this.interval *= 0.7;\n",
       "    if(this.direction > 0){this.play_animation();}\n",
       "    else if(this.direction < 0){this.reverse_animation();}\n",
       "  }\n",
       "\n",
       "  Animation.prototype.anim_step_forward = function()\n",
       "  {\n",
       "    this.current_frame += 1;\n",
       "    if(this.current_frame < this.frames.length){\n",
       "      this.set_frame(this.current_frame);\n",
       "    }else{\n",
       "      var loop_state = this.get_loop_state();\n",
       "      if(loop_state == \"loop\"){\n",
       "        this.first_frame();\n",
       "      }else if(loop_state == \"reflect\"){\n",
       "        this.last_frame();\n",
       "        this.reverse_animation();\n",
       "      }else{\n",
       "        this.pause_animation();\n",
       "        this.last_frame();\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "\n",
       "  Animation.prototype.anim_step_reverse = function()\n",
       "  {\n",
       "    this.current_frame -= 1;\n",
       "    if(this.current_frame >= 0){\n",
       "      this.set_frame(this.current_frame);\n",
       "    }else{\n",
       "      var loop_state = this.get_loop_state();\n",
       "      if(loop_state == \"loop\"){\n",
       "        this.last_frame();\n",
       "      }else if(loop_state == \"reflect\"){\n",
       "        this.first_frame();\n",
       "        this.play_animation();\n",
       "      }else{\n",
       "        this.pause_animation();\n",
       "        this.first_frame();\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "\n",
       "  Animation.prototype.pause_animation = function()\n",
       "  {\n",
       "    this.direction = 0;\n",
       "    if (this.timer){\n",
       "      clearInterval(this.timer);\n",
       "      this.timer = null;\n",
       "    }\n",
       "  }\n",
       "\n",
       "  Animation.prototype.play_animation = function()\n",
       "  {\n",
       "    this.pause_animation();\n",
       "    this.direction = 1;\n",
       "    var t = this;\n",
       "    if (!this.timer) this.timer = setInterval(function(){t.anim_step_forward();}, this.interval);\n",
       "  }\n",
       "\n",
       "  Animation.prototype.reverse_animation = function()\n",
       "  {\n",
       "    this.pause_animation();\n",
       "    this.direction = -1;\n",
       "    var t = this;\n",
       "    if (!this.timer) this.timer = setInterval(function(){t.anim_step_reverse();}, this.interval);\n",
       "  }\n",
       "</script>\n",
       "\n",
       "<div class=\"animation\" align=\"center\">\n",
       "    <img id=\"_anim_imgSWGHEUCATPXKHBPP\">\n",
       "    <br>\n",
       "    <input id=\"_anim_sliderSWGHEUCATPXKHBPP\" type=\"range\" style=\"width:350px\" name=\"points\" min=\"0\" max=\"1\" step=\"1\" value=\"0\" onchange=\"animSWGHEUCATPXKHBPP.set_frame(parseInt(this.value));\"></input>\n",
       "    <br>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.slower()\">&#8211;</button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.first_frame()\"><img class=\"anim_icon\" src=\"\"></button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.previous_frame()\"><img class=\"anim_icon\" src=\"\"></button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.reverse_animation()\"><img class=\"anim_icon\" src=\"\"></button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.pause_animation()\"><img class=\"anim_icon\" src=\"\"></button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.play_animation()\"><img class=\"anim_icon\" src=\"\"></button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.next_frame()\"><img class=\"anim_icon\" src=\"\"></button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.last_frame()\"><img class=\"anim_icon\" src=\"\"></button>\n",
       "    <button onclick=\"animSWGHEUCATPXKHBPP.faster()\">+</button>\n",
       "  <form action=\"#n\" name=\"_anim_loop_selectSWGHEUCATPXKHBPP\" class=\"anim_control\">\n",
       "    <input type=\"radio\" name=\"state\" value=\"once\" > Once </input>\n",
       "    <input type=\"radio\" name=\"state\" value=\"loop\" checked> Loop </input>\n",
       "    <input type=\"radio\" name=\"state\" value=\"reflect\" > Reflect </input>\n",
       "  </form>\n",
       "</div>\n",
       "\n",
       "\n",
       "<script language=\"javascript\">\n",
       "  /* Instantiate the Animation class. */\n",
       "  /* The IDs given should match those used in the template above. */\n",
       "  (function() {\n",
       "    var img_id = \"_anim_imgSWGHEUCATPXKHBPP\";\n",
       "    var slider_id = \"_anim_sliderSWGHEUCATPXKHBPP\";\n",
       "    var loop_select_id = \"_anim_loop_selectSWGHEUCATPXKHBPP\";\n",
       "    var frames = new Array(0);\n",
       "    \n",
       "  frames[0] = \"\"\n",
       "  frames[1] = \"\"\n",
       "  frames[2] = \"\"\n",
       "  frames[3] = \"\"\n",
       "  frames[4] = \"\"\n",
       "  frames[5] = \"\"\n",
       "  frames[6] = \"\"\n",
       "  frames[7] = \"\"\n",
       "  frames[8] = \"\"\n",
       "  frames[9] = \"\"\n",
       "  frames[10] = \"\"\n",
       "  frames[11] = \"\"\n",
       "  frames[12] = \"\"\n",
       "  frames[13] = \"\"\n",
       "  frames[14] = \"\"\n",
       "  frames[15] = \"\"\n",
       "  frames[16] = \"\"\n",
       "  frames[17] = \"\"\n",
       "  frames[18] = \"\"\n",
       "  frames[19] = \"\"\n",
       "  frames[20] = \"\"\n",
       "  frames[21] = \"\"\n",
       "  frames[22] = \"\"\n",
       "  frames[23] = \"\"\n",
       "  frames[24] = \"\"\n",
       "  frames[25] = \"\"\n",
       "  frames[26] = \"\"\n",
       "  frames[27] = \"\"\n",
       "  frames[28] = \"\"\n",
       "  frames[29] = \"\"\n",
       "  frames[30] = \"\"\n",
       "  frames[31] = \"\"\n",
       "  frames[32] = \"\"\n",
       "  frames[33] = \"\"\n",
       "  frames[34] = \"\"\n",
       "  frames[35] = \"\"\n",
       "  frames[36] = \"\"\n",
       "  frames[37] = \"\"\n",
       "  frames[38] = \"\"\n",
       "  frames[39] = \"\"\n",
       "  frames[40] = \"\"\n",
       "  frames[41] = \"\"\n",
       "  frames[42] = \"\"\n",
       "  frames[43] = \"\"\n",
       "  frames[44] = \"\"\n",
       "  frames[45] = \"\"\n",
       "  frames[46] = \"\"\n",
       "  frames[47] = \"\"\n",
       "  frames[48] = \"\"\n",
       "  frames[49] = \"\"\n",
       "  frames[50] = \"\"\n",
       "  frames[51] = \"\"\n",
       "  frames[52] = \"\"\n",
       "  frames[53] = \"\"\n",
       "  frames[54] = \"\"\n",
       "  frames[55] = \"\"\n",
       "  frames[56] = \"\"\n",
       "  frames[57] = \"\"\n",
       "  frames[58] = \"\"\n",
       "  frames[59] = \"\"\n",
       "  frames[60] = \"\"\n",
       "  frames[61] = \"\"\n",
       "  frames[62] = \"\"\n",
       "  frames[63] = \"\"\n",
       "  frames[64] = \"\"\n",
       "  frames[65] = \"\"\n",
       "  frames[66] = \"\"\n",
       "  frames[67] = \"\"\n",
       "  frames[68] = \"\"\n",
       "  frames[69] = \"\"\n",
       "  frames[70] = \"\"\n",
       "  frames[71] = \"\"\n",
       "  frames[72] = \"\"\n",
       "  frames[73] = \"\"\n",
       "  frames[74] = \"\"\n",
       "  frames[75] = \"\"\n",
       "  frames[76] = \"\"\n",
       "  frames[77] = \"\"\n",
       "  frames[78] = \"\"\n",
       "  frames[79] = \"\"\n",
       "  frames[80] = \"\"\n",
       "  frames[81] = \"\"\n",
       "  frames[82] = \"\"\n",
       "  frames[83] = \"\"\n",
       "  frames[84] = \"\"\n",
       "  frames[85] = \"\"\n",
       "  frames[86] = \"\"\n",
       "  frames[87] = \"\"\n",
       "  frames[88] = \"\"\n",
       "  frames[89] = \"\"\n",
       "  frames[90] = \"\"\n",
       "  frames[91] = \"\"\n",
       "  frames[92] = \"\"\n",
       "  frames[93] = \"\"\n",
       "  frames[94] = \"\"\n",
       "  frames[95] = \"\"\n",
       "  frames[96] = \"\"\n",
       "  frames[97] = \"\"\n",
       "  frames[98] = \"\"\n",
       "  frames[99] = \"\"\n",
       "  frames[100] = \"\"\n",
       "  frames[101] = \"\"\n",
       "  frames[102] = \"\"\n",
       "  frames[103] = \"\"\n",
       "  frames[104] = \"\"\n",
       "  frames[105] = \"\"\n",
       "  frames[106] = \"\"\n",
       "  frames[107] = \"\"\n",
       "  frames[108] = \"\"\n",
       "  frames[109] = \"\"\n",
       "  frames[110] = \"\"\n",
       "  frames[111] = \"\"\n",
       "  frames[112] = \"\"\n",
       "  frames[113] = \"\"\n",
       "  frames[114] = \"\"\n",
       "  frames[115] = \"\"\n",
       "  frames[116] = \"\"\n",
       "  frames[117] = \"\"\n",
       "  frames[118] = \"\"\n",
       "  frames[119] = \"\"\n",
       "  frames[120] = \"\"\n",
       "  frames[121] = \"\"\n",
       "  frames[122] = \"\"\n",
       "  frames[123] = \"\"\n",
       "  frames[124] = \"\"\n",
       "  frames[125] = \"\"\n",
       "  frames[126] = \"\"\n",
       "  frames[127] = \"\"\n",
       "  frames[128] = \"\"\n",
       "  frames[129] = \"\"\n",
       "  frames[130] = \"\"\n",
       "  frames[131] = \"\"\n",
       "  frames[132] = \"\"\n",
       "  frames[133] = \"\"\n",
       "  frames[134] = \"\"\n",
       "  frames[135] = \"\"\n",
       "  frames[136] = \"\"\n",
       "  frames[137] = \"\"\n",
       "  frames[138] = \"\"\n",
       "  frames[139] = \"\"\n",
       "  frames[140] = \"\"\n",
       "  frames[141] = \"\"\n",
       "  frames[142] = \"\"\n",
       "  frames[143] = \"\"\n",
       "  frames[144] = \"\"\n",
       "  frames[145] = \"\"\n",
       "  frames[146] = \"\"\n",
       "  frames[147] = \"\"\n",
       "  frames[148] = \"\"\n",
       "  frames[149] = \"\"\n",
       "  frames[150] = \"\"\n",
       "  frames[151] = \"\"\n",
       "  frames[152] = \"\"\n",
       "  frames[153] = \"\"\n",
       "  frames[154] = \"\"\n",
       "  frames[155] = \"\"\n",
       "  frames[156] = \"\"\n",
       "  frames[157] = \"\"\n",
       "  frames[158] = \"\"\n",
       "  frames[159] = \"\"\n",
       "  frames[160] = \"\"\n",
       "  frames[161] = \"\"\n",
       "  frames[162] = \"\"\n",
       "  frames[163] = \"\"\n",
       "  frames[164] = \"\"\n",
       "  frames[165] = \"\"\n",
       "  frames[166] = \"\"\n",
       "  frames[167] = \"\"\n",
       "  frames[168] = \"\"\n",
       "  frames[169] = \"\"\n",
       "  frames[170] = \"\"\n",
       "  frames[171] = \"\"\n",
       "  frames[172] = \"\"\n",
       "  frames[173] = \"\"\n",
       "  frames[174] = \"\"\n",
       "  frames[175] = \"\"\n",
       "  frames[176] = \"\"\n",
       "  frames[177] = \"\"\n",
       "  frames[178] = \"\"\n",
       "  frames[179] = \"\"\n",
       "  frames[180] = \"\"\n",
       "  frames[181] = \"\"\n",
       "  frames[182] = \"\"\n",
       "  frames[183] = \"\"\n",
       "  frames[184] = \"\"\n",
       "  frames[185] = \"\"\n",
       "  frames[186] = \"\"\n",
       "  frames[187] = \"\"\n",
       "  frames[188] = \"\"\n",
       "  frames[189] = \"\"\n",
       "  frames[190] = \"\"\n",
       "  frames[191] = \"\"\n",
       "  frames[192] = \"\"\n",
       "  frames[193] = \"\"\n",
       "  frames[194] = \"\"\n",
       "  frames[195] = \"\"\n",
       "  frames[196] = \"\"\n",
       "  frames[197] = \"\"\n",
       "  frames[198] = \"\"\n",
       "  frames[199] = \"\"\n",
       "\n",
       "\n",
       "    /* set a timeout to make sure all the above elements are created before\n",
       "       the object is initialized. */\n",
       "    setTimeout(function() {\n",
       "        animSWGHEUCATPXKHBPP = new Animation(frames, img_id, slider_id, 50, loop_select_id);\n",
       "    }, 0);\n",
       "  })()\n",
       "</script>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Imports specifically so we can render outputs in Colab.\n",
    "from matplotlib import animation\n",
    "from JSAnimation.IPython_display import display_animation\n",
    "from IPython.display import display\n",
    "\n",
    "\n",
    "def display_frames_as_gif(frames):\n",
    "    \"\"\"Displays a list of frames as a gif, with controls.\"\"\"\n",
    "    patch = plt.imshow(frames[0])\n",
    "    plt.axis('off')\n",
    "\n",
    "    def animate(i):\n",
    "        patch.set_data(frames[i])\n",
    "\n",
    "    anim = animation.FuncAnimation(\n",
    "        plt.gcf(), animate, frames = len(frames), interval=50\n",
    "    )\n",
    "    display(display_animation(anim, default_mode='loop'))\n",
    "    \n",
    "        \n",
    "# display \n",
    "display_frames_as_gif(frames)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
